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ABSTRACT 


• — In  the  parsT  year ,  eur  'research  on  optical  pattern  recognition  (OPR)  for 
missile  guidance  has  addressed  three  general  areas:  new  OPR  techniques,  OPR 
experiments  on  specific  missile  guidance  problems  and  scenarios,  and  component 

evaluation  and  testing.  In  the  area  of  component  evaluation  and  testing,  ire- 

-U-Ul 

•have-  concentrated  on  2-D  spatial  light  modulators  (SLM)  with  specific  attention 
to  the  photo-DKDP,  prom,  and  the  liquid  crystal  light  valve  and  to  a  general 
test  and  evaluation  procedure  for  these  devices.  The- major  restrirts  of  this 
.program,  phase  has  been  the  development  of  a  general  SLM  test  and  evaluation 
procedure^  0ur—in_hojjs£^tes^  and  evaluation  program  fcronxh  the  photo-DKDP  and 
HCD  addressed  SLMs  worthy  of  future  research  and  development.  In  the  area  of 
new  OPR  techniques,  vn  have  developed  a  coherence  measure  correlator,  an  equal¬ 
ization  correlator j  moment  invariant  hybrid  correlator  and  an  averaged  filter 
correlator^  Specific  missile  guidance  scenarios  persued  were  OPR  of  multi-sensor 
and  infrared  imagery.  The  former  has  laid  initial  ground  work  for  a  future 
program  in  optical  statistical  and  deterministic  pattern  recognition.  The  latter 
has  initiated  a  generalized  approach  to  OPR  using  hyperspace  formulations  and 
linear  combinations  of  orthonormal  basis  functions .>/ 
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1.  INTRODUCTION 

In  the  past  year,  our  research  on  OPR  for  missile  guidance  has  concentrated 
on  three  general  areas:  (1)  component  evaluation,  (2)  new  OPR  techniques,  and 
(3)  demonstration  of  OPR  on  several  specific  missile  guidance  data  bases. 

As  the  most  critical  OPR  system  components  we  correctly  choose  the  2-D 
real-time  and  reusable  SLM.  In  the  course  of  the  past  year,  we  have  performed 
in  house  evaluation  of  the  liquid  crystal  light  valve  (LCLV)  and  the  photo-DKDP 
SLMs.  The  results  of  our  general  SLM  program  are  summarized  in  chapter  2.  In 
this  paper,  we  address  the  general  test  and  evaluation  procedure  to  be  used  for 
these  components.  Such  an  effort  is  vital  if  comparisons  and  fair  evaluations 
of  the  diverse  candidate  SLMs  are  to  be  possible.  Many  detailed  experiments 
wer,e  performed  on  the  photo-DKDP  SLM  in  many  theoretical  studies  of  this  device 
were  performed  in  conjunction  with  researchers  in  France.  The  results  of  these 
tests  are  summarized  in  chapter  3.  We  found  this  SLM  to  be  one  of  the  most 
promising  ones,  with  considerable  development  research  possible  that  should 
result  in  a  viable  device.  The  results  of  our  in  house  evaluation  of  several 
LCLV  SLMs  are  included  in  chapter  2.  The  major  flaw  in  this  SLM  appears  to_be 
its  spatially  varying  nonuniformity  of  response  due  to  small  differences  in  the 
thickness  of  the  LC  layer.  These  results  and  details  of  the  associated  theory 
will  comprise  a  portion  of  our  next  annual  report. 

OPR  techniques  have  classicaiy  relied  on  the  matched  spatial  filter  (MSP) 
and  the  frequency  plane  correlator  (FPC) .  We  have  considered  and  developed 
several  alternatives  to  this  system  over  the  past  year.  A  hybrid  coherent 
system  described  In  chapter  A  is  a  modification  of  the  joint  transform  correla¬ 
tor  in  which  the  second  Fourier  transform  is  performed  digitally.  The  resultant 
system  is  a  coherence  measure  correlator.  It  is  of  immense  use  in  1-D  pattern 
recognition  such  as  for  Tercom  in  conjunction  with  a  phase  lock  loop  output 
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sensor.  A  modification  of  this  system  (the  equalization  correlator)  allows  one 
to  correct  for  different  known  space-invariant  degrading  point  spread  functions 
in  the  correlation.  The  third  new  OPR  technique  we  have  developed  is  the  use 
of  a  weighted  MSF  synthesis  system  to  combine  the  edge  enhancement  preprocessing 
step  (found  to  be  of  use  in  multi-sensor  image  pattern  recognition)  and  MSF 
correlation  into  a  single  one  step  processor.  This  research  is  summarized  in 
chapter  5.  The  fourth  new  OPR  technique  we  have  developed  is  the  use  of  an 
average  MSF  formed  from  a  linear  combination  of  orthonormal  basis  functions. 

In  this  system,  the  problem  of  immediate  concern  is  the  recognition  of  an  object 
independent  of  its  orientation  using  an  average  filter.  This  research  is 
summarized  in  chapter  6.  The  fifth  and  final  new  OPR  technique  we  have  performed 
initial  research  on  in  the  past  year  involves  a  hybrid  optical/digital  system 
to  compute  the  absolute  normalized  invariant  moments  of  an  object.  Computation 
of  these  feature  vectors  by  digital  techniques  can  then  provide  the  desired 
pattern  recognition  information  on  the  presence  of  an  object  independent  of  its 
geometric  orientation.  The  initial  results  of  this  research  are  summarized  in 
chapter  7. 

The  final  phase  of  our  research  on  the  use  of  OPR  techniques  for  missile 
guidance  involves  the  test  and  evaluation  of  various  OPR  methods  on  divers 
AFOSR  image  data  bases  and  missile  guidance  scenarios.  In  chapter  3,  we  briefly 
discuss  the  results  of  our  research  on  real-time  SLMs  for  visible  image  pattern 
recognition.  In  chapter  4,  uses  made  of  imagery  from  WPAFB.  In  chapter  5,  a 
multi-sensor  missile  guidance  image  pattern  recognition  problem  is  described  and 
the  results  of  tests  performed  on  a  data  base  consisting  of  visible  and  radar 
scenes  are  presented.  In  chapter  6,  we  utilize  a  high  resolution  infrared  image 
data  base  of  key  objects,  specifically  tanks,  in  a  terminal  guidance  pattern 
recognition  problem.  In  all  of  these  scenarios,  with  new  OPR  techniques  and 
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refinements  in  existing  techniques,  successful  pattern  recognition  was  obtained 
by  optical  computing  systems.  These  systems  are  most  promising  for  future  advance 
missile  guidance  applications. 

A  summary  paper  of  coherent  OPR  research  is  included  as  chapter  8.  A  list 
of  the  14  papers  published  in  the  past  year  and  22  talks  presented  in  the  past 
year  on  our  AFOSR  sponsored  research  are  listed  in  chapters  9  and  10. 


CHAPTER  2 


PERFORMANCE  EVALUATION  OF  SPATIAL  LIGHT  MODULATORS 


Reprinted  from  APPLIED  OPTICS,  Vol.  18,  page  2445,  July  15, 1979 
Copyright  ©  1979  by  the  Optical  Society  of  America  and  reprinted  by  permission  of  the  copyright  owner. 


Performance  evaluation  of  spatial  light  modulators 


David  Casasent 


An  experimental  test  procedure  and  associated  theoretical  formulations  are  described  for  the  evaluation  of 
spatial  light  modulators  (SLMs)  for  coherent  optical  computing.  The  experimental  data  included  were  ob¬ 
tained  by  us  on  Hughes  liquid  crystal  and  photo  DKDP  SLMs  among  others.  However,  in  this  paper  we  ad¬ 
dress  only  the  general  procedures  to  be  used  to  evaluate  SLM  performance.  The  detailed  aspects  of  these 
tests  and  their  relationships  to  specific  SLMs  will  be  covered  in  separate  papers. 


I.  Introduction 

Optical  computing1  offers  the  attractive  features 
of  parallel  processing  in  real  time  and  thus  has  been  of 
considerable  research  interest.  It  has  long  been  rec¬ 
ognized  that  to  realize  these  advantages  of  such  systems, 
real-time,  reusable,  2-D  input  and  often  filter  plane 
transducers  are  essential.  These  components  are  re¬ 
ferred  to  as  spatial  light  modulators  (SLMs).2-4  Our 
present  concern  will  be  with  the  experimental  proce¬ 
dures  we  feel  should  be  used  in  the  evaluation  of  SLMs. 
By  this  approach  we  hope  to  clarify  many  of  the  mis¬ 
conceptions  present  concerning  the  use  of  SLMs,  in¬ 
terpreting  the  published  specifications  for  these  SLMs, 
devising  appropriate  acceptance  tests  for  these  SLMs, 
and  the  procedures  to  be  used  to  perform  these  tests. 

The  appropriateness  of  various  tests  depends  on  and 
varies  with  the  SLM  used  as  well  as  the  intended  ap¬ 
plication.  Space  does  not  permit  an  extensive  discus¬ 
sion  of  all  SLMs2'4  or  of  all  optical  computing  applica¬ 
tions,1  and  certainly  not  extensive  discussions  of  when 
each  test  is  appropriate.  The  reader  will  thus  have  to 
be  content  with  general  remarks  (the  purpose  of  this 
paper).  Experimental  results  of  various  test  methods 
applied  to  the  Hughes  hybrid  field-effect  liquid  crystal,5 
photo  KD*P  (deuterated  potassium  dihydrogen  phos¬ 
phate),6'8  and  the  Prom9  light  valves  will  be  included. 
The  discussions  will  be  such  that  general  conclusions 
and  procedures  to  be  used  emerge  that  are  of  use  in 
general  SLM  evaluation.  Specific  discussions  of  these 
data  for  each  individual  SLM  will  be  discussed  in  future 
papers. 
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Brief  descriptions  of  the  three  SLMs  used  are  pro¬ 
vided  in  Sec.  II  for  completeness  and  to  enable  future 
issues  to  be  better  understood.  We  divide  SLM  per¬ 
formance  criteria  into  three  areas:  sensitometry  (Sec. 
Ill);  modulation  transfer  function  (MTF)  (Sec.  IV);  and 
noise  (Sec.  V).  This  present  paper  concerns  only  op¬ 
tically  addressed  electrooptical  SLMs.  More  detailed 
extensions  to  electron  beam  addressed  SLMs  and  to 
phase  modulated  SLMs  can  be  made  and  will  be  de¬ 
tailed  later. 

II.  Electrooptical  Optically  Addressed  SLMs 

The  three  major  devices  that  are  representative  of 
this  class  of  SLM1  are  the  LCLV  (liquid  crystal  light 
valve),5  photo  KD*P,6  8  and  the  Prom.9  The  LCLV 
consists  of  jver  twenty  thin  film  and  other  layers.  It 
basically  consists  of  a  photoconductor,  mirror,  light¬ 
blocking,  and  LCLV  active  material  layers  between  two 
outer  transparent  electrodes.  In  operation,  an  ac 
voltage  is  applied  between  the  electrodes,  and  the  pat¬ 
tern  to  be  recorded  is  incident  on  the  photoconductor 
at  A  iy,  whereas  readout  is  in  reflection  at  XR  from  the 
LCLV  side  of  the  device. 

The  structure  and  operation  of  photo  KD*P  are 
similar  except  no  light  blocking  layer  exists,  and  a  dc 
voltage  is  present  across  the  electrodes.  Other  differ¬ 
ences  are  that  the  polarity  and  value  of  this  voltage  are 
changed  during  the  write  (W),  read  (R),  and  erase  (E) 
cycles.  The  major  difference  between  the  LCLV  and 
the  other  SLMs  is  the  lack  of  long  term  storage  in  the 
LCLV  and  the  need  for  active  erase  mechanisms  (apply 
the  proper  voltage  to  the  electrodes  and  flooding  the 
photoconductor  with  light  of  the  proper  X)  in  the  photo 
KD*P  and  Prom.  Normally  Xw  =*  514  nm  and  XR  » 
633  nm.  In  all  cases,  readout  through  crossed  polarizers 
is  required. 

The  Prom  differs  from  the  prior  SLMs  in  that  the 
electrooptic  crystal  is  both  electrooptical  and  photo- 
conductive.  Since  no  photoconductor  layer  is  present, 
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Fig.  1.  Global  sensitometry  curves  showing  variation  with  voltage 
applied  across  the  SLM. 


insulating  parylene  layers  are  used  to  produce  the  re¬ 
quired  spatial  voltage  division  across  the  active  material 
proportional  to  the  input  Aw  irradiance.  It  is  generally 
available  only  in  the  transmission  rather  than  the  re¬ 
flection  readout  configuration.  Even  though  no  di¬ 
electric  mirror  is  present  within  the  SLM  structure,  it 
is  still  generally  used  in  the  reflex  readout  mode. 

III.  Sensitometry 

The  representatives  of  the  optically  addressed  class 
of  SLMs  are  all  sensitive  to  either  the  input  light  irra¬ 
diance  (/)  or  the  light  energy  (£).  The  first  data  re¬ 
quired  for  any  SLM  is  a  sensitometry  curve  of  Uw  (in¬ 
cident  irradiance  at  Aw)  vs  amplitude  (£ )  or  irradiance 
(t)  transmittance.  The  former  curve  (Uw  vs  t )  applies 
when  coherent  light  is  used  for  readout,  whereas  the 
latter  curve  (/xw  vs  r)  is  of  use  in  projection  display  or 
similar  applications  when  readout  uses  noncoherent 
light.  For  the  LCLV,  both  the  voltage  V  applied  be¬ 
tween  electrodes  and  its  frequency  f  affect  sensitometry. 
Such  data  are  shown  in  Fig.  1  for  the  LCLV.  From 
these  data,  we  see  that  the  linearity  of  response,  the 
dynamic  range,  the  useful  Uw  exposure  range,  and  the 
Uw  threshold  at  which  the  device  responds  all  vary  with 
V  and  /.  For  the  Prom  and  photo  KD*P  only  V  is 
variable  because  a  dc  voltage  is  applied. 

We  have  found  that  V  and  /  also  affect  the  SLM  noise 
and  the  spatial  uniformity  of  the  SLM’s  response.  We 
have  also  found  considerable  difference  between  global 
sensitometry  data  (Fig.  1)  (obtained  by  illuminating  and 
reading  a  large  SLM  area)  and  local  sensitometry  data 
(Fig.  2)  (obtained  over  a  small  region  of  the  SLM). 
Response  time,  reciprocity  data,  and  other  information 
on  the  SLM’s  dynamic  response  are  also  of  concern  and 
often  vary  considerably  from  static  data  and  vary  also 
with  the  Aw  illumination  range  used  and  the  address 
time  per  point. 

A  schematic  of  the  basic  sensitometry  measurement 
system  used  is  shown  in  Fig.  3.  The  vertically  polarized 
Ar  light  from  a  He-Ne  laser  incident  on  the  polarizing 
beam  splitter  (PBS)  is  reflected  toward  the  SLM.  With 
Uw  =  0,  the  Ar  light  reflected  from  the  SLM  is  also 
vertically  polarized  and  Ur  -  0.  As  Uw  increases,  the 
elliptical  polarization  of  the  A r  light  reflected  from  the 


SLM  increases  and  so  does  the  reflected  /xr  component 
that  passes  through  the  PBS.  To  obtain  local  sensi¬ 
tometry  data,  the  spatial  variation  of  such  data,  and  the 
effect  of  V  and  /  choices  on  noise,  linearity,  and  dynamic 
range,  we  used  apertures  A\  and  A2  shown  in  Fig.  3. 

For  the  LCLV,  we  illuminated  the  photoconductor  side 
of  the  LCLV  through  only  one  of  the  3-mm2  regions  of 
A  i  and  measured  I\r  in  all  nine  3-mm  regions  of  A2  as 
V,  f,  and  /xw  were  varied.  We  denote  Ikr  in  the  cor¬ 
responding  3-mm2  region  of  A2  as  signal  and  the  Ur  ’ 
value  in  all  other  regions  of  A  2  as  noise. 

A  summary  of  our  results  for  the  LCLV  follow.  Less 
noise  was  observed  as  /  was  decreased  and  V  was  in-  < 
creased.  As  /  was  lowered,  the  noise  tracked  the  signal 
at  a  lower  rate.  Most  important  were  the  large  spatial 
differences  we  found  in  the  voltage  Vr  and  irradiance 
Uwt  threshold  differences  (at  which  Ur  changed). 
Large  differences  in  V?  and  Uwr  for  signal  and  noise 
are  desirable.  For  the  LCLV,  we  found  VV  to  be  lower 
at  low  /  with  larger  signal  and  noise  thresholds  at  {  =  1  j 

kHz  than  at  10  kHz  and  with  noise  tracking  the  signal  i 
slower  at  lower  f  values.  The  local  sensitometry  data  ! 


Fig.  2.  Representative  local  sensitometry  curve. 


Fig.  II.  Schematic  diagram  of  the  SLM  global  and  local  sensitometry 
data  acquisition  system. 
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for  a  3-mm2  region  of  the  LCLV  were  shown  in  Fig.  2. 
The  large  1500:1  contrast  ratio  and  30,000:1  dynamic 
range  shown  in  Fig.  2  are  misleading,  since  spatial  sen- 
sitometry  variations  make  the  global  values  (Fig.  1)  for 
such  parameters  much  less.  We  have  found  these 
spatial  variations  to  be  equivalent  to  horizontal  shifts 
of  the  sensitometry  curve  or  effectively  to  changes  in 
I\wt  due  to  spatial  variations  in  the  photoconductor, 
CdTe/CdS  heterojunction  diode,  and  LC  thickness 
variations.  Thus  only  global  sensitometry  data  should 
be  used,  with  local  sensitometry  data  being  of  use  in 
determining  the  reasons  for  the  obtained  performance 
and  the  spatial  response  uniformity  of  the  device. 

The  importance  of  SLM  sensitometry  data  as  a  first 
step  in  SLM  testing  and  other  aspects  of  such  data  can 
be  seen  from  Fig.  1.  From  Fig.  1,  we  see  that  the  dy¬ 
namic  range  of  the  SLM  increases  with  V.  For  our 
application,  a  large  linear  dynamic  range  was  desirable, 
thus  V  =  4-6  V  and  /  =  10  kHz  were  chosen  as  the  SLM 
operating  values.  Other  applications  may  require 
emphasis  of  other  SLM  features.  Previously  pub¬ 
lished10  LCLV  sensitometry  data  exhibited  a  knee  in 
the  curve  (as  in  the  8-V  curve  in  Fig.  1).  However,  as 
shown,  proper  selection  of  V  and  /  can  result  in  a  linear 
t-E  curve.  This  feature  by  which  the  shape  of  the 
SLM’s  sensitometry  curve  can  be  controlled  is  of  po¬ 
tential  use  in  various  applications  such  as  2-D  optical 
analog/digital  conversion,  halftone  screening,  homo¬ 
morphic  filtering,  etc. 

From  Fig.  1,  we  find  the  global  and  hence  useful  linear 
dynamic  range  {IxW  max/hw  min)  of  the  SLM  to  be 
about  1000:1,  the  contrast  ratio  1  max/I\n  min  to  be 
70:1,  and  the  desirable  lxw  bias  level  (center  of  the 
linear  portion  of  the  curve)  to  be  70-100  nW/cm2. 

The  photodetector  used  to  measure  I\n  places  a  lower 
limit  on  these  curves.  For  our  case,  IXr  min  was  70 
nW/cm2,  tmin  was  0.78%,  and  Tmin  was  0.6%.  The  de¬ 
tector  used  in  such  sensitometry  tests  and  the  I\w  range 
used  should  of  course  be  compatible  with  the  levels 
present  during  the  SLM’s  intended  application.  The 
22%  transmission  loss  observed  (tmax  =  78%,  Tmax  = 
60%)  was  attributed  to  incomplete  reflections  from  the 
dielectric  mirror  within  the  LCLV  structure,  to  nonideal 
alignment  of  the  LCLV  molecules,  and  to  the  inability 
of  the  device  to  rotate  completely  vertically  polarized 
input  Xr  light  into  horizontally  polarized  A/(  reflected 
light.  Lens  and  PBS  losses  have  been  compensated  for 
in  the  data  of  Figs.  1  and  2. 

Because  of  the  hybrid  field  effect  used  in  the  LCLV, 
the  Ar  wavelength  used  must  be  matched  to  the  thick¬ 
ness  of  the  LC  layer  to  optimize  contrast  ratio.1112  For 
other  SLMs,  A«  must  be  chosen  to  reduce  the  destruc¬ 
tive  effects  of  reading  on  a  stored  image.  For  other 
SLMs  operated  with  dc  voltages,  we  have  found  the 
dielectric  mirrors  to  pass  considerable  Aw  light  thus 
degrading  a  stored  pattern  while  it  is  being  read.  Al¬ 
though  readout  through  a  crossed  polarizer/analyzer  or 
PBS  system  is  theoretically  optimum,1112  we  have 
found  the  use  of  separate  polarizers  and  analyzers  ori¬ 
ented  slightly  off  from  90°  to  increase  results  and  the 
flexibility  of  the  measurement  system. 


For  integrating  or  storage  mode  SLMs  such  as 
photo-DKDP,  we  have  found  that  extra  care  should  be 
taken  in  acquiring  sensitometry  data.  Specifically,  in 
the  photo-DKDP  light  valve,  we  have  observed  a  strong 
dependence  of  the  photosensitivity  of  the  Se  photo¬ 
conductor  to  the  potential  gradient  present  across  it. 
As  a  result,  when  long  exposure  times  and  low  /*w  light 
levels  are  used,  the  SLM’s  sensitivity  to  the  initial  and 
later  photocarriers  differs  considerably.  In  such  cases, 
one  can  obtain  very  different  sensitometry  data  de¬ 
pending  upon  the  write  light  exposure  time  used.  As 
usual  and  here  especially,  the  test  conditions  should 
duplicate  those  present  in  the  SLM’s  actual  applica¬ 
tion. 

IV.  Modulation  Transfer  Function  (MTF) 

The  MTF  of  an  SLM  or  any  device  used  in  an  optical 
processor  is  one  of  the  most  important  but  seemingly 
least  understood  or  most  misused  parameters.  We  have 
analytically  and  experimentally  evaluated  over  eight 
different  MTF  tests.  Only  the  most  useful  ones  are 
discussed  here.  The  appropriateness  of  various  MTF 
tests  depends  upon  the  specific  SLM  and  its  intended 
system  application.  A  discussion  of  MTF  and  defini¬ 
tions  might  appear  too  tutorial,  but  is  vital  to  provide 
an  adequate  framework  for  the  MTF  data  to  follow. 

A.  Theoretical  Formulation 

We  assume  the  pattern  recorded  on  the  SLM  to  be  a 
sinusoid  at  spatial  frequency  u'  (in  1  -D  for  simplicity). 
We  first  clearly  distinguish  between  amplitude  mA  and 
irradiance  mi  modulation  by  describing  the  SLM’s 
transmittance  by 

t(x)  =  (1  +  mA  cos27ru'x).  (la) 

t(jc)  =  (1  +  cos2iru'jr);i,  (lb) 

where  t  and  r  denote  amplitude  and  irradiance  trans¬ 
mittance,  respectively.  If  a  reconstructed  image  of  a 
test  pattern  recorded  on  the  SLM  is  formed  using  the 
SLM  as  a  relay  and  the  detected  output  pattern  is 
scanned,  the  measured  modulation  is 

^  W/  —  (tmax  tmin)/(tmax  *b  tmin).  (2) 

where  Imax  and  7mjn  are  the  measured  quantities  (the 
maximum  and  minimum  irradiances  in  the  recon¬ 
structed  image).  A  binary  image  such  as  the  3-bar  Air 
Force  resolution  chart  is  used  as  the  test  pattern  in  such 
a  test. 

If  noncoherent  illumination  is  used  in  the  above 
imaging  modulation  test,  the  detected  pattern  (that  is 
scanned  to  determine  /max  and  /mjn)  is  r.  If  the  image 
reconstruction  is  performed  in  coherent  light,  the  de¬ 
tected  pattern  is  |t|2.  Since  r  =  |t|2,  no  difference 
should  result,  and  m  =  m/  defined  by  Eq.  (2)  is  ob¬ 
tained.  In  practice,  the  MTF  and  OTF  of  the  associ¬ 
ated  imaging  optics  used  affect  the  observed  output 
patterns  in  the  coherent  and  noncoherent  illumination 
cases,  respectively.  Since  the  MTF  of  a  system  is  su¬ 
perior  to  its  OTF,  coherent  illumination  is  recom¬ 
mended  for  any  such  imaging  test  (when  the  modulation 
of  the  SLM  alone  rather  than  of  the  entire  system,  in- 
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eluding  lenses,  is  desired).  We  have  consistently  ob¬ 
tained  higher  modulation  values  when  coherent  rather 
than  noncoherent  illumination  was  used. 

Amplitude  modulation  m,(  is  the  desired  MTF  pa¬ 
rameter  in  coherent  optical  processing  application.  We 
can  relate mA  tom/  as 


mi 


2mA 
1  + 


(3) 


which  follows  directly  from  the  definitions  in  Eqs.  (1) 
and  (2). 

However,  this  relationship  is  valid  only  for  monotone 
inputs,  and  m;  is  best  obtained  in  a  coherent  image  re¬ 
construction  experiment.  We  can  always  obtain  m/ 
from  mA  using  Eq.  (3). 

It  is  also  possible  and  often  preferable  to  measure  m 
from  the  Fourier  transform  of  Eq.  (1).  For  this  case,  we 
use  t  in  Eq.  (la)  to  describe  the  transmittance  of  the 
SLM.  The  amplitude  of  the  Fourier  transform  of  Eq. 
(la)  is 


T(u)  =  A0|5(u)  +  (mA/2)6(u  ±  u')j.  (4) 

The  irradiances  / o  and  //  of  the  dc  and  each  first-order 
term  are  easily  measurable  in  the  Fourier  plane.  From 
Eqs.  (1)— (4),  these  are  related  to  m  and  the  measurable 
quantity  /j//0  by 


Uh  =  <mA)2/4, 

mA  =  2(/,//0)‘« 

4  </, //„>*« 

mi  = - 

1  +  4  U  illa) 


(5a) 

(5b) 

(5c) 


tions  in  noncoherent  readout  light,  an  unnormalized 
irradiance  MTF  curve  of  mi  vs  u'  is  appropriate.  Ifm 
is  measured  in  an  image  plane,  Eq.  (2)  is  used;  if  the 
measurement  is  made  in  a  Fourier  plane,  Eq.  (5)  is  used. 
In  practice,  MTF  as  defined  is  really  a  contrast  transfer 
function  (since  the  phase  of  the  imagery  is  neglected). 
However,  the  above  definitions  have  become  common 
of  late  and  relate  directly  to  the  intended  device  pa¬ 
rameter  we  wish  to  describe  and  quantify.  We  will  thus 
still  refer  to  a  plot  of  m  vs  u  as  MTF. 

For  deformable  SLMs  and  phase  modulators,  the 
MTF  has  a  bandpass  response  and  m)lUt(0)  =  0  and  m 
=  0  if  t  is  described  by  Eq.  ( 1).  Thus  MTF  as  described 
by  Eq.  (6)  is  not  usable  for  such  devices.  Rather,  the 
diffraction  efficiency  >7  of  these  devices  is  specified.  For 
these  SLMs,  77  is  the  product  of  the  fraction  of  the  input 
light  in  one  first-order  usable  for  modulation  and  the 
amount  of  average  input  light  that  can  be  diffracted  into 
one  first-order.  Thus 

my 4  1  +  m\/ 2  _  m2A 

1  +  mJj/2  (l+mA)2  4(1 +  mA)2 

_  |1  ~  (1  -  miA),n\2  ml. 

(1  +  mA  +  (1  -  m^)l/2l2  16  '  J 

In  Eq.  (9),  mn  is  defined  by  Eq.  (6),  and  mjn  =  1  and 
m„ut(0)  =  1  are  assumed.  The  final  approximation  in 
Eq.  (9)  is  usually  made.  This  is  only  valid  for  small  m„ 
and  is  not  appropriate  since  tj  =  >/16  when  mn  =  1. 
However,  Eq.  (9)  is  still  used  to  extrapolate  MTF  from 
V  data  since  low  17  values  look  more  impressive  when 
expressed  as  an  MTF. 


The  classic  MTF  as  defined  is  a  plot  of  normalized 
modulation  m„: 

mtM(uVmin(u') 

m„  = - - -  (6) 

m„ut(O)/min(0) 

vs  the  input  spatial  frequency  u'.  The  measurable 
quantity  in  Eqs.  (5)  is  m,lul.  It  generally  decreases  as 
u'  increases.  One  generally  normalizes  m  by  dividing 
it  by  its  maximum  value  mout(0),  which  occurs  at  low  or 
zero  spatial  frequency  (for  deformable  SLMs  with 
bandpass  MTFs,  alternate  normalizations  are  required). 
Further  normalization  by  mm(u')/mm(0)  is  used  when 
the  modulation  min  of  the  recorded  pattern  is  not  100% 
and  when  mjn  varies  with  the  recorded  spatial  frequency 
u'.  Thus,  for  this  classic  MTF  plot,  mn  varies  from  1 
to  0  as  u'  increases  from  0  spatial  frequency. 

For  reasons  we  amplify  later,  we  have  found  the  use 
of  an  unnormalized  m  vs  u'  plot  to  be  preferable.  We 
thus  plot  mvsu',  where 

"W<u'> 

m  = - - - •  (7a) 

min(u')/m  in(0) 

For  coherent  optical  systems,  an  unnormalized  ampli¬ 
tude  MTF  mA  vs  u'  is  used.  Generally,  we  advocate 
that  normalization  by  the  input  modulation  not  be  used, 
rather  a  plot  of 

m  =  m„„,(u')  (7b) 

vs  spatial  frequency  u'  be  provided  and  the  constant 
input  modulation  be  specified.  For  imaging  applica- 


B.  Sensitometry  Effects 

Three  aspects  of  SLM  sensitometry  affect  MTF. 
From  Figs.  1  and  2,  we  see  that  the  device’s  I region 
used  should  be  linear  if  linear  modulation  is  desired  and 
should  cover  a  large  range  if  optimum  modulation  is  of 
concern.  T o  achieve  either  result,  control  of  the  average 
I kw  avg  write  light  and/or  the  input  bias  light  level 
/ \w  bias  is  needed.  The  recording  mechanism  used  to 
address  the  SLM  in  its  intended  actual  application  and 
the  type  of  input  data  to  be  processed  affect  the  bias 
level  and  contrast  ratio  that  will  actually  be  present  in 
the  input  data.  For  example,  reflective  aerial  imagery 
usually  has  a  higher  bias  level  and  lower  contrast  than 
other  imagery,  whereas  signal  processing  applications 
usually  demand  a  large  dynamic  range  and  linear  re¬ 
cording. 

A  second  effect  of  SLM  sensitometry  arises  when 
linear  recording  (a  linear  relationship  between  t  and 
I\w)  is  of  concern  (e.g.,  as  in  many  signal  processing 
applications).  In  these  cases,  use  of  the  proper  region 
of  the  device’s  sensitometry  curve  is  vital.  The  most 
appropriate  method  we  have  found  for  obtaining  linear 
MTF  data  is  interferometrically  (Sec.  IV.D)  by  mea¬ 
suring  the  second -order  Fourier  term  (I2)  as  well  as  /« 
and  1\.  By  adjusting  the  input  bias  and  modulation 
levels  until  I2  is  20  dB  below  /j ,  we  can  insure  that  in¬ 
terharmonic  distortion  is  below  1%;  we  then  note  that 
a  1%  linear  MTF  results.  Other  I \/I2  ratios  yield  MTF 
data  with  other  degrees  of  linearity. 
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The  effect  of  gamma  7  (the  slope  of  the  SLMs  t  - 
I\w  curve  on  log-log  axes)  on  MTF  is  the  final  effect  of 
sensitometry  on  MTF  that  we  consider.  To  include  this 
effect,  we  assume  an  irradiance  transmittance  for  the 
SLM  of  the  form 

t(x)  =  r„(l  I-  mi  cos2tii,x)~'>.  (10) 

The  Fourier  transform  of  the  binomial  expansion  of  t 
=  r 1/2  can  be  manipulated  to  relate  / 1//«  to  m/  and  y  by 
Ref.  9: 

m,  =  (4/>)(/ ,//(i)l/2.  (11) 

From  Eq.  (11),  we  see  that  an  SLM  with  7  =  1  yields 
twice  the  irradiance  MTF  of  an  SLM  with  7  =  2.  In 
most  cases,  it  is  inappropriate  to  separate  y  and  MTF 
effects,  except  when  comparing  theoretical  and  exper¬ 
imental  resolution  data. 

C.  Imaging  MTF 

The  most  used  and  in  some  sense  the  simplest  mea¬ 
sure  of  SLM  resolution  and  MTF  is  obtained  by  imaging 
a  resolution  chart  (such  as  the  3-bar  Air  Force  test 
chart)  onto  the  SLM,  reconstructing  the  image  of  the 
test  pati-':  ii  using  the  SLM  as  a  relay  device,  and  then 
scanning  the  reconstructed  image  with  a  photometric 
microscope.  We  then  measure  /mjn  and  /max  for  each 
group/element  in  the  reconstructed  image  of  the  test 
chart;  m„ul(u')  is  thus  obtained  from  Eq.  (2),  and  an 
MTF  curve  defined  by  Eq.  (7b)  not  Eq.  (6)  is  plotted. 
mA  or  mi  is  used  depending  upon  the  application. 

Examples  of  several  experimentally  obtained  un¬ 
normalized  imaging  MTF  curves  for  the  LCLV  at  dif¬ 
ferent  voltages  and  frequencies  are  shown  in  Fig.  4. 
The  modulation  of  the  imaging  optics  used  was  mea¬ 
sured  to  be  unity,  whereas  mm  for  the  test  chart  was 
found  to  be  0.9  out  to  80  cycles/mm.  The  MTF  data 
presented  have  been  normalized  for  these  values  as  in 
Eq.  (7b).  These  initial  MTF  curves  serve  to  demon¬ 
strate  several  vital  aspects  of  MTF  data.  First  and 
foremost,  unnormalized  MTF  data  must  be  used  since 
m  is  then  directly  the  fraction  of  the  input  light  that  is 
usable  for  modulation.  To  demonstrate  this,  we  select 
the  m  =  0.4  point  on  both  curves.  This  corresponds  to 
some  given  amount  of  usable  output  light.  From  Fig. 
4(a),  we  find  that  we  can  obtain  this  amount  of  light  for 
an  input  spatial  frequency  of  up  to  19  cycles/mm; 
whereas  for  the  V  and  f  settings  used  in  the  Fig.  4(b) 
data,  we  can  obtain  this  amount  of  light  only  up  to  13 
cycles/mm  of  input  data.  If  both  curves  were  normal¬ 
ized,  such  comparative  data  would  not  be  present. 

Another  obvious  remark  concerning  data  acquisition 
is  that  the  MTF  data  should  be  continued  to  low  enough 
spatial  frequencies  that  three  approximately  equal  and 
constant  modulation  values  result.  This  is  of  utmost 
importance  if  the  MTF  curve  is  normalized.  The  third 
issue  concerns  the  SLM  resolution  one  reads  from  such 
data.  The  limiting  SLM  resolution  at  m  =  0.05  is  often 
given  but  is  rarely  appropriate  for  use  because  of  the 
negligible  light  levels  available  at  this  low  modulation. 
Only  a  light  budget  analysis  for  the  full  actual  system 
can  answer  what  SLM  modulation  is  necessary.  We 


advocate  the  use  of  unnormalized  MTF  data  and  that 
the  resolution  be  given  at  the  m  =  0.5  or  50%  modulation 
point.  In  addition,  we  note  that  the  input  hw  bias 
value  and  the  true  input  modulation  used  be  provided 
if  any  MTF  data  are  to  be  of  use  at  the  system  level. 
Examples  of  such  data  follow  in  Sec.  IV. D. 

Other  peculiarities  of  MTF  data  depend  on  the  spe¬ 
cific  SLM  being  tested.  For  the  LCLV,  we  obtained 
MTF  curves  for  various  control  voltages  and  ac 
frequencies  and  found  the  MTF  to  be  better  at  higher 
(10-kHz)  frequencies.  This  appears  to  be  due  to  the 
increase  in  the  ratio  of  the  transverse-to-longitudinal 
impedance  of  the  LCLV  with  frequency.  Another 
specific  feature  of  the  LCLV  is  the  effect  that  the  Ar 
choice  has  on  the  device’s  performance.  To  reduce  the 
off-state  transmittance  of  the  SLM,  the  thickness  of  the 
LC  layer  must  be  properly  adjusted,  because  the  twisted 
nematic  effect  determines  the  off  state.  The  thickness 
of  the  LC  layer  in  the  LCLV  we  used  was  optimized  for 
Ar  =  633  nm.  We  can  partly  attribute  the  lower  mod¬ 
ulation  and  resolution  obtained  in  Fig.  4(b)  compared 
with  Fig.  4(a)  to  the  broadband  Ar  light  used  in  the  one 
case.  Other  MTF  tests  performed  on  the  LCLV  with 


Kin.  4.  Imaging  MTF  curves:  (a)OVnnd  10  kHz  with  coherent  Ar 
=  633  nm  light;  (b)  3.5  V  and  10  kHz  with  noncoherent  filtered  Ar  = 
620  ±  10-nm  light. 
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Aft  =  514  nm  yielded  even  poorer  modulation  and  res¬ 
olution. 

Our  objections  to  an  imaging  MTF  test  are  many. 
The  test  is  subjective  and  plagued  by  operator  attitude 
and  error.  This  is  due  primarily  to  the  degree  to  which 
the  input  data  can  be  focused  onto  the  SLM.  More 
important,  since  the  test  patterns  used  (3-bar  Air  Force 
chart)  are  binary,  this  type  of  test  does  not  investigate 
the  gray  scale  resolution  or  linearity  of  the  SLM.  Fi¬ 
nally,  the  input  bias  light  level  and  input  modulation 
are  not  easily  variable  in  such  MTF  tests. 

D.  Interferometric  MTF 

For  the  reasons  noted  above,  interferometric  MTF 
data  are  preferable  to  imaging  MTF  data  for  reasons  of 
both  experimental  ease  and  data  content.  The  inter¬ 
ferometric  MTF  data  acquisition  system  used  is  shown 
in  Fig.  5.  In  this  architecture,  the  interference  pattern 


Fig.  5.  Schematic  diagram  of  an  SLM  interferometric  MTF  data 
acquisition  system. 


SPATIAL  mourner  Indtt/mml 

Fig.  6.  Nonlinear  interferometric  MTF  curves  at  different  bias  and 
input  modulation. 


Fig.  7.  Linear  interferometric  MTF  curve  with  1%  SLM  linearity. 


of  two  plane  waves  of  irradiances  lA  and  /«  inclined  at 
an  angle  0  is  recorded.  The  subsequent  amplitude 
transmittance  is 


/  =  Ia  +  /«  +  2(/a/«l,/2cos‘2iru'i.  (121 

where  u'  =  47r(sin0)/A  is  the  spatial  frequency  of  the 
recorded  fringe  pattern  in  cycles/mm.  From  Eqs.  ( 1 ), 
(12),  and  (2),  we  find 


1 +  </*//,.)  !+/<’ 


(t:» 


From  Eq.  (13),  we  see  that  by  varying  the  ratio  R  of  the 
intensities  of  the  two  beams,  we  can  control  m.  By 
controlling  /aVK  =  l a  +  /«,  we  can  vary  the  bias  point 
chosen  on  the  SLM’s  sensitometry  curve.  By  varying 
0  by  changing  the  tilt  angle  of  mirror  M:u  the  spatial 
frequency  u'  can  be  changed.  The  Fourier  transform 
of  Eq.  (13)  can  then  be  scanned  by  a  fiber  optic  probe 
(FP),  photometric  microscope,  and  photomultiplier 
(PMT)  system. 

This  system  topology  of  Fig.  5  thus  allows  us  to  vary 
easily  the  bias  level,  spatial  frequency  u\  and  rnodula 
tion  m  of  the  input  fringe  pattern.  Of  further  concern 
is  the  fact  that  the  sine  wave  pattern  in  Eq.  <  131  is  au¬ 
tomatically  in  focus  over  the  entire  thickness  of  the 
SLM,  thus  completely  overcoming  any  input  focusing 
errors  that  could  arise  in  an  imaging  MTF  system.  To 
obtain  the  MTF  of  the  SLM.  we  can  monitor  -«nd  / 1 
in  the  Fourier  transform  of  Eq.  (13)  for  dilferen.  u 
input  spatial  frequencies  and  apply  Eq.  <5bl  or  Eq  <5c> 
to  determine  m.  Equation  (7b)  is  then  used  to  obtain 
the  MTF  plot  of  mvsu'. 

Two  such  MTF  curves  are  shown  in  Fig.  »>  with 
slightly  different  m  jn  and  /avK  levels.  These  two  curves 
vividly  demonstrate  many  of  the  MTF  remarks  ad 
vanced  earlier.  The  effect  of  a  slight  1 8%  change  in  ni 
on  MTF  is  seen  to  be  dramatic.  With  a  lower  m  „„  less 
of  the  SLM’s  sensitometry  curve  is  used  and  the  reduced 
m  shown  results.  The  need  to  specify  MTF  test  con¬ 
ditions  (/aVg  and  mm)  is  thus  apparent.  Similarly,  only- 
through  unnormalized  MTF  data,  as  shown,  do  these 
differences  appear.  In  the  lower  curve  in  Fig.  6,  we  find 
a  lower  peak  m  (due  to  the  lower  m  in);  but  the  resolution 
(at  50%  of  the  peak  m  value)  is  found  to  be  larger  (37 
cycles/mm)  for  the  lower  mjn  =  0.82  case  than  for  the 
larger  mjn  =  1.0  case  (32  cycles/mm).  However,  the 
unnormalized  MTF  data  of  Fig.  6  clearly  convey  the 
point  that  more  usable  output  light  is  always  present 
with  the  larger  mjn  value.  Such  results  would  not  ap¬ 
pear  in  MTF  data  if  the  normalized  MTF  definition  in 
Eq.  (6)  were  used.  Similarly,  the  relevance  of  these  data 
would  not  be  of  use  unless  /aV|!  and  min  were  specified 
as  done  in  Fig.  6. 

We  refer  to  the  data  of  Fig.  6  as  a  nonlinear  MTF 
curve.  A  linear  MTF  curve  for  the  same  LCLV  SLM 
is  shown  in  Fig.  7.  To  obtain  these  latter  data,  l  >  was 
monitored  as  well  as  /,//„.  By  keeping  /2  below  /,  by 
20  dB  (by  reducing  m in  and  adjusting  /avR  accordingly), 
we  can  ensure  that  the  SLM’s  resultant  MTF  is  linear 
within  1%  (i.e.,  less  than  1%  interharmonic  distortion). 
With  /avg  adjusted  to  lie  in  the  center  of  the  linear  por- 


2450  APPLIED  OPTICS  /  Vol.  18,  No.  14  /  15  July  1979 


tion  of  the  SLM’s  sensitometry  curve  of  Fig.  1  and  with 
min  reduced  to  0.79,  the  curve  of  Fig.  7  results.  The 
resolution  at  m  =  0.25  (50%  of  the  peak  m  =  0.5  value) 
is  seen  to  be  32  cycles/mm.  The  modulation  at  32  cy¬ 
cles/mm  is  below  that  obtainable  for  nonlinear  SLM 
operation  in  Fig.  6  as  expected.  Only  such  fully  docu¬ 
mented  and  unnormalized  MTF  data  are  of  use  in  de¬ 
ciding  on  the  performance  to  be  expected  of  an  SLM  in 
a  given  application  as  well  as  in  comparing  the  MTFs 
of  different  SLMs. 

The  effect  of  modulation  on  the  performance  of  an 
optical  correlator  is  rarely  quantified,  but  is  of  major 
concern  in  determining  the  usable  SLM  resolution  in 
a  system  scenario.  For  example,  an  irradiance  modu¬ 
lation  of  10%  is  equivalent  to  an  amplitude  modulation 
of  5%.  The  resultant  peak  intensity  in  one  sidelobe  in 
the  Fourier  plane  pattern  is  32  dB  below  dc.  This 
represents  an  intolerable  loss  in  most  cases.  Such  light 
level  budget  analyses  are  needed  for  each  specific  ap¬ 
plication  to  determine  the  necessary  system  perfor¬ 
mance.  The  output  detector  used  greatly  affects  such 
system  data. 


E.  Depth  of  Focus 

Interferometric  MTF  data  are  more  reproducible 
than  imaging  MTF  data  and  include  gray  scale  and 
linearity  information  on  the  SLM.  It  has  thus  become 
the  most  used  MTF  data  acquisition  method.  How¬ 
ever,  because  of  the  infinite  depth  of  field  of  the  re¬ 
corded  fringe  pattern  (the  feature  that  makes  this  MTF 
method  easy  to  perform),  this  test  may  yield  different 
results  than  one  would  obtain  in  the  SLM’s  actual  sys¬ 
tem  application.  The  actual  depth  of  focus  of  the  in¬ 
terferometric  fringe  pattern  is  effectively  the  thickness 
of  the  photoconductor  layer  (10  ^im)  for  photo- KD*P 
and  the  LCLV.  However,  the  Prom  crystal  is  thick 
(500-900  pm)  and  both  electrooptic  and  photocon- 
ductive.  For  such  devices,  the  depth  of  focus  of  the 
interferometric  pattern  is  effectively  the  crystal’s 
thickness. 

Interferometric  MTF  data  will  be  valid  only  if  the 
SLM’s  actual  data  recording  system  has  a  depth  of  focus 
comparable  with  the  thickness  of  the  photoconductor 
or  crystal  (depending  upon  the  SLM).  In  addition,  for 
an  SLM  such  as  photo-KD*P  with  a  10-pm  thick  pho¬ 
toconductor  layer,  interferometric  MTF  data  are  only 
valid  up  to  50-cycle/mm  input  spatial  frequencies  (the 
reciprocal  of  2X10  pm)  and  only  if  the  depth  of  focus 
of  the  data  recording  system  in  the  device’s  actual  ap¬ 
plication  exceeds  10  pm.  When  the  SLM  is  used  in  the 
frequency  plane  and  interference  patterns  such  as  a 
matched  spatial  filter15  or  joint  transform14  are  recorded 
on  it,  the  interferometric  MTF  is,  of  course,  valid. 
When  the  SLM  is  used  as  an  image  plane  transducer, 
the  infinite  depth  of  focus  assumption  is  not  always 
valid.  If  low  /-number  optics  are  used,  care  must  be 
taken  that  the  thickness  of  the  photoconductor  does  not 
exceed  the  depth  of  focus  of  the  actual  recording  optics 
used.  For /-number  optics  with  a  10-pm  spot  size,  the 
depth  of  focus  is 


depth  of  focus  =  2  (/-numberKspot  size)  =  80  pm.  (14) 

If  the  photoconductor’s  thickness  (or  the  active  crystal 
thickness  for  the  case  of  the  From)  exceeds  this  80-pm 
value,  interferometric  MTF  test  data  are  not  appro¬ 
priate.  In  actual  use,  the  MTF  will  decrease  because 
of  overlap  of  the  light  cone  away  from  focus.  In  sum¬ 
mary,  interferometric  MTF  data  will  yield  the  best 
possible  SLM  resolution  and  modulation.  Whether 
such  SLM  performance  is  actually  obtainable  depends 
upon  the  specific  way  the  SLM  is  used  in  the  actual 
system  application. 

F.  Alternate  MTF  Tests 

The  usefulness,  ease  of  implementation,  and  repro¬ 
ducibility  of  over  eight  MTF  tests  were  evaluated 
first-hand  on  over  four  different  SLMs  during  this 
program.  MTF  tests  that  required  sine  wave  input 
transparencies  of  various  spatial  frequencies  and  LFM 
Sayce  charts  of  linearly  varying  spatial  frequency  were 
found  to  suffer  from  lack  of  inputs  of  adequate  preci¬ 
sion.  Correlation-based  MTF  tests15  were  found  to  be 
plagued  by  low  light  levels  and  diffraction  efficiency 
(i.e.,  the  problem  became  one  of  measuring  detector 
accuracy  rather  than  SLM  performance). 

One  attractive  MTF  test  that  is  appropriate  for 
evaluation  of  SLMs  in  system  applications  in  which 
data  are  written  sequentially  on  the  SLM  by  a  scanning 
recording  device  is  the  edge  or  gradient  MTF  test 
method.16'18  In  such  a  test,  an  edge  is  imaged  onto  the 
SLM,  and  its  reconstructed  image  (using  the  SLM  as 
a  relay  device),  is  scanned.  These  data  are  then 
smoothed  and  can  be  corrected  for  the  SLM’s  sensi¬ 
tometry  curve  if  appropriate.  Differentiation  of  such 
an  edge  image  results  in  the  SLM’s  line  spread  function. 
The  Fourier  transform  of  this  line  spread  function  is 
then  the  SLM’s  optical  transfer  function,  whose  mag¬ 
nitude  is  the  device’s  MTF.  The  operations  of 
smoothing,  sensitometry  correction,  differentiation, 
Fourier  transformation,  and  magnitude  calculation  on 
the  edge  image  are  generally  performed  in  digital  elec¬ 
tronics.  This  results  in  a  rather  indirect  MTF  calcu¬ 
lation.  This  fact  plus  the  requirement  that  the  system 
be  space-invariant  in  the  image  analysis  domain  (optical 
systems  are  in  fact  space-variant)19  21  tend  to  reduce 
the  potential  appropriateness  of  this  MTF  analysis 
method.  In  time,  dedicated  hardware  and  software 
advances  may  change  this  observation. 


V.  Noise 

The  third  and  final  aspect  of  SLM  evaluation  to  be 
discussed  is  that  of  SLM  noise.  We  distinguish  four 
types  of  noise:  distortions;  phase;  and  scatter  and  sig¬ 
nal-dependent  noise. 

A.  Distortion 

Distortion  is  of  concern  when  linear  recording  is  re¬ 
quired.  MTF  data  with  control  of  the  interharmonic 
distortion  level  can  be  measured  by  the  linear  inter¬ 
ferometric  MTF  technique  described  in  Sec.  IV. D. 
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(a)  (b) 

Fig.  8.  Interferograms  of  the  OPD  of  an  SUM:  (aid  V;(bl  5  V,  6 
kHz. 


B.  Phase  Error 

Phase  errors  are  of  utmost  concern  in  coherent  optical 
correlators.  Slowly  varying  phase  errors  or  linear  phase 
errors  are  not  of  major  concern  except  in  optical  corre¬ 
lators.  Such  error  sources  in  coherent  optical  com¬ 
puters  are  best  described  by  optical  path  difference 
(OPD)  maps.21'24  Such  OPD  plots  must  be  provided 
for  all  input  spatial  frequencies  and  aperture  sizes  to 
characterize  completely  a  coherent  optical  system.22 
To  relate  these  phase  errors  to  the  accuracy  of  an  optical 
processor,  2-D  spatial  OPD  maps  are  necessary.2’ 
Techniques  for  easily  acquiring  such  data  have  been 
previously  reported.22-24  Accurate  analysis  of  such  data 
requires  one  to  model  the  optical  processor  and  its 
components  as  a  space-variant  system.19'21  Thus,  to 
obtain  quantitatively  meaningful  SLM  phase  error  data, 
one  must  acquire  spatial  OPD  data  on  the  SLM  rather 
than  the  rms  OPD  data  characteristically  available  for 
such  components.  In  summary,  it  is  not  the  average 
phase  error  that  matters,  but  rather  how  this  phase  error 
is  distributed. 

In  obtaining  any  type  of  SLM  data,  even  rms  OPD, 
the  te9t  should  be  performed  with  voltage  applied  to  the 
SLM  [Fig.  8(b)J,  since  often  a  considerably  different 
OPD  phase  error  results  than  if  no  voltage  is  applied  to 
the  device  [Fig.  8(a)]. 

C.  Scatter  Level 

The  surface  roughness  and  cosmetic  imperfections 
of  the  SLM  generally  manifest  themselves  as  random 
phase  errors  of  high  spatial  frequency.  These  noise 
sources  predominantly  contribute  to  the  system’s 
background  noise  level  or  scatter  level.  We  refer  to  this 
as  scatter  level  noise.  It  directly  affects  the  maximum 
obtainable  system  dynamic  range.  The  extent  of  this 
noise  source  is  most  easily  measured  by  imaging  a 
square  aperture  onto  the  SLM  and  scanning  its  Fourier 
transform  pattern.  Such  a  cross-sectional  Fourier  scan 
is  shown  in  Fig.  9(b).  For  comparison  purposes,  the 
Fourier  transform  of  an  empty  square  aperture  obtained 
using  the  same  lens  system  is  scanned  [Fig.  9(a)].  Care 
should  be  taken  to  extend  the  Fourier  plane  scan  out  to 
several  hundred  sidelobes  until  the  pattern’s  expo¬ 
nential  decrease  stabilizes. 

From  Fig.  9(a),  we  find  that  the  scatter  level  of  the 
optical  system  alone  [Fig.  9(a)|  exceeds  40  dB  at  u  =  4 


cycles/mm  and  is  within  1  dB  of  the  theoretical  value. 
Comparing  Figs.  9(a)  and  9(b),  we  see  that  the  presence 
of  the  SLM  (the  LCLV  for  this  data)  results  in  an  in¬ 
creased  scatter  level  3-5  dB  above  the  theoretical  value 
and  above  that  of  the  optical  system  alone. 

D.  Signat-to-Noise  Ratio  (SNR) 

Signal  dependent  noise  or  the  ratio  of  signal  to  noise 
level  (in  the  same  spatial  frequency  band)  is  often  of 
more  concern.  We  have  found  two  methods  to  be  of  use 
in  obtaining  such  data.  The  first  is  to  record  a 
square-wave  pattern  on  the  device  by  imaging  an  ac¬ 
curate  test  pattern  onto  the  SLM.  A  scan  of  the  Fourier 
transform  of  this  input  square  wave  is  shown  in  Fig.  10 
for  a  5-cycle/mm  square  wave  recorded  on  the  LCLV. 
This  pattern  shows  Fourier  components  at  both  even 
and  odd  harmonics  of  the  fundamental  frequency  of  the 
square- wave  grating.  The  amplitudes  of  these  Fourier 
components  are  all  within  1  dB  of  the  theoretical  values 
found  from  a  Fourier  series  expansion  of  the  input 
pattern.  Our  present  concern  is  with  the  measurable 
null  depth  of  this  pattern.  From  Fig.  10,  we  see  that  the 
system’s  dynamic  range,  including  all  SLM  noise,  is  a 
maximum  of  52  dB  even  with  an  input  signal  present. 
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Fig.  9.  Cross-sectional  scan  of  the  Fourier  transform  of  a  square 
aperture  for  scatter-level  noise  tests:  t at  empty  aperture;  (lit  SLM 
in  aperture. 
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Fig.  10.  SNR  SLM  test  data:  (a)  cross-sectional  scan  <if  the  Fourier 
transform  of  a  square  wave  recorded  on  the  SLM;  (bl  plot  of  SNR  vs 
spatial  frequency  obtained  from  Fourier  plane  data  of  interfemme- 
trically  recorded  sine  waves.1' 


Since  the  amplitude  of  increasing  Fourier  orders  of 
the  square  wave  decreases,  whereas  the  noise  level  re¬ 
mains  essentially  constant,  true  SNR  is  not  obtainable 
from  such  data.  However,  true  SNR  can  be  measured 
by  interferometrically  recording  a  sine  wave  on  the  SLM 
and  investigating  its  Fourier  transform  pattern  for 
various  input  spatial  frequencies  u'.  The  maximum 
obtainable  l\Ui  ratio  or  the  I\  to  scatter  level  ratio  ob¬ 
tainable  in  the  Fourier  plane  will  then  provide  us  with 
the  SNR  of  the  SLM  vs  spatial  frequency  u'.  An  ex¬ 
ample  of  such  data  is  shown5  in  Fig.  10(b)  for  the  LCLV. 
The  increase  in  SNR  with  spatial  frequency  occurs  be¬ 
cause  the  system’s  scatter  level  decreases  faster  with  u 
than  the  SLM’s  MTF.  Care  should  be  taken  to  mea¬ 
sure  such  l\,  1 1  and  scatter  levels  with  an  area  photo¬ 
diode  detector  whose  size  equals  the  diffraction  limited 
spot  size  of  the  Fourier  transform  system  used. 


VI.  Conclusions 

As  optical  computing  and  SLM  technology  mature 
and  as  more  coherent  optical  processing  systems  are 
fabricated  and  delivered,  the  specifications  and  testing 
of  the  SLM  components  (so  vital  to  such  systems)  must 
be  standardized.  We  feel  the  procedure  described  in 
this  paper  and  demonstrated  by  experimental  confir¬ 
mation  provide  a  necessary  first  step  toward  such 
ends. 
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Abstract 


The  photo-DKDP  spatial  light  modulator  is  found  to  be  an  excellent  candidate  real-time  and  reusable  2-D 
optically-addressed  spatial  light  modulator.  Extensive  sensitometry ,  MTF,  and  other  device  data  are  reported 
with  emphasis  on  a  new  field  dependent  photo-sensitivity  and  the  importance  of  linear  MTF  data.  A  new  spatial 
birefringent  modulation  transfer  function  is  described  and  its  use  in  obtaining  more  linear  device  response, 
bias  level  suppression,  phase  modulation  and  1  msec  erase  time  are  noted.  New  real-time  image  processing, 
optical  pattern  recognition,  and  optical  signal  processing  applications  of  this  SLM  are  described  with  empha¬ 
sis  on  the  accuracy  of  the  photo-DKDP  based  real-time  experiments. 

1.  Introduction 


The  photo-DKDP  spatial  light  modulator  (SLM)  has  been  inexistence  for  six  years.  It  is  a  most  promising 
real-time  and  reusable  2-D  opticaLlv  addressed  SLM  for  coherent  optical  data  processing.  Although  it  is 
basically  well  understood  and  extensively  documented, man v  details  of  its  sensitometry  and  MTF  data  require 
additional  description.  These  new  issues  include  a  charge  carrier  description  of  the  device  (Section  ))  ,  the 
importance  of  the  ut  product  and  voltage  dependent  sensitometry  of  the  Se  photo-conductor  on  charge  generation 
in  the  device  (Section  4),  and  its  linear  grav  scale  MTF  and  use  of  the  proper  MTF  data  accuisition  techniques 
(Section  5).  Other  SLM  performance  specifications  such  as  storage,  optical  quality,  and  scatter  level  are 
noted  in  Section  6.  A  new  poLarization  description  of  the  spatial  birefringent  modulation  in  this  SLM  leads 
to  new  and  more  linear  device  response  and  to  new  write  and  erase  methods  with  the  abilitv  to  suppress  input 
bias  levels  and  achieve  erase  times  below  1  msec  (Section  7).  These  new  photo-DKDP  device  physics  and  device 
operation  issues  are  followed  by  the  highlights  of  several  new  application  experiments  performed  using  this 
SLM  (Section  3).  These  include  image  subtraction,  optical  pattern  recognition,  and  optical  signal  processing, 
with  emphasis  on  comparison  of  film  based  and  theoretically  expected  results  to  real-time  experimentally  ob¬ 
tained  ones. 


2.  Device  construction  and  operation 

The  photo-DKDP  SLM  Is  shown  schematically  in  Figure  1  and  detailed  fabrication  issues  are  included  in 
Table  1.  The  general  operation  of  this  device  is  well-known , but  it  summarized  here  for  completeness.  The 
SLM  is  optically-addressed  by  a  spatially  modulated  input  light  distribution  or  a  sequentially  modulated 
scanning  light  pattern.  The  device  is  written  on  in  Xy  light  (usually  440-514nm)  from  the  Se  photo-conductor 
side  and  read  and  reflection  in  light  (usually  633nm)  from  the  DKDP  side.  In  its  normal  raoae  of  operation, 
the  readout  light  1^  is  linearly  polarized  along  the  y  axis  (at  45°  to  the  crystal's  induced  x 1  and  y*  elec¬ 
tro-optic  axes).  The  modulated  readout  light  intensity  Iq  is  monitored  through  a  crossed  linear  polarizer 
oriented  along  the  x  axis. 


Table  1.  Fabrication  of  the  photo-DKDP  SLM 


Component 


Specif icat ion 


El  Gold 

Se  12um,  n  *  2.3  at  633nm,  e  *  6 


M  9  layers  X/4  of  CaF^  and  ZnS.  T^  3  0.985 

and  T  *  0.001  at  633nm 

DKDP  Wedged  (320-Oum),  35x28mm",  ec  ■  650  and 

\  3  300v  at  T  -  Tc  *  0.5*  and  633nm 

In->03,  Tj  ■  0.98,  T^  3  0.01,  T^  3  0.01  at  633nm 

Substrate  CaF,  wedged  (6  -  5.17  mm),  1°30' 


Readout  window  SiOj,  wedged  3° 
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After  writing,  a  spatial  voltage  distribution  V  Is  pr  duced  across  the  DKDP  Layer  proportional  to  the  spa¬ 
tial  w^ite  light  Input  distribution  I^.  By  the  Poekels ^effect ,  the  output  light  intensity  Iq  (through  a 
crossed  polarizer)  is  related  to  the  applied  voltage  byJ 

I0  -  Ii  sin-  ( (V/V,_)ir )  ,  (1) 

where  Vij  is  the  crystal's  halfwave  voltage.  The  DKDP  is  held  at  its  Curie  temperature  of  approximately  -52°C 
by  Peltier  cells  around  Che  circumference  of  the  device.  This  decreases  the  halfwave  voltage  to  approximately 
300  volts  and  greatly  improves  the  device's  resolution,  uniformity  and  storage  properties. 2,6, 7 

3.  Charge  carrier  description 

A  description  of  the  operation  of  the  photo-DKDP  SLM  in  terms  of  the  hole  and  electron  charge  carriers  is 
necessary  to  fully  analyze  and  describe  the  sensitometry ,  readout,  and  erase  operation  of  this  SLM.  Such  a 
description  has  not  previously  been  provided  and  in  fact  the  applied  voltage  polarity  noted  in  many  of  the 

papers  describing  this  SLM  are  conflicting.  In  Figure  2,  we  show  the  voltage  distribution  (vertical)  vs.  dis¬ 
tance  (horizontal)  across  the  various  device  layers  for  the  four  different  states  of  device  operation  for  both 

bright  (BR)  and  dark  (DK)  (illuminated  and  non-illuminated)  regions  of  the  device. 

tnitially,  +Vq  is  applied  (with  positive).  With  no  I^w,  the  voltage  distribution  is  as  shown  in  Figure 
2a  (most  of  the  applied  Vq  appears  the  Se  photo-conductor  Laver).  When  the  Se  photo-conductor  is  illuminated 
with  Ay  light,  hole  and  electron  charge  carriers  are  generated.  With  *  +Vq,  the  hoLes  conduct  and  move  to 
the  mirror  M,  decreasing  the  voltage  at  M  and  increasing  the  voltage  drop  across  the  DKDP  In  3R  areas  (see 
Figure  2b).  When  the  electrodes  E^  and  E2  are  shorted  (Vq  *  0) ,  a  large  +V|  appears  across  the  DKDP  In  BR 
areas  and  a  smaller  +V2  In  DK  areas.  With  crossed  output  polarizers,  a  positive  output  image  results.  During 
readout,  no  major  carrier  conduction  occurs  because  M  blocks  the  A^  light  and  the  Se  Is  much  less  sensitive  to 
Che  A^  wavelength  light  chat  leaks  through  M.  For  erasure,  the  Se  side  of  the  SLM  is  uniformly  flooded  with 
Ac  light  (in  the  wavelength  region  Ay)  with  Vq  *  0.  This  Ag  illumination  generates  holes  and  electrons  in 
the  Se.  This  causes  the  holes  trapped  at  M  during  writing  to  migrate  back  to  E^;  equivalently,  the  electrons 
at  can  migrate  to  M.  Both  processes  decrease  the  positive  voltage  at  M  and  thus  remove  the  writing 
charge  scored  there. 

We  terra  the  above  process  positive  writing.  Writing  with  -VQ  on  E^  is  also  possible  and  is  refered  to  as 
negative  writing.  The  charge  carrier  description  of  operation  is  similar  for  this  case.  We  note  that  in 
positive  writing,  writing  is  by  hole  conduction  and  erasure  by  electron  conduction.  Conversely,  in  negative 
writing,  writing  is  by  electron  flow  and  erasure  is  by  hole  flow.  This  is  of  concern  since  hole  conduction 
is  better  than  electron  conduction  in  Se.  A  new  feature  of  this  SLM  that  we  will  discuss  in  Section  4  is  the 
Increase  in  the  conduction  efficiency  and  photo-sensitivitv  of  the  Se  when  a  field  is  present  across  the  Se. 
These  and  other  issues  contribute  to  the  reasons  for  the  new  write  and  erase  schemes  to  be  described  in  Sec¬ 
tion  7.  The  different  fields  present  across  the  Se  in  the  different  cases  (Figure  2)  are  thus  of  importance 
when  analyzing  the  different  device  operating  modes.  For  readout,  OV  is  preferable  for  best  contrast.  To 
see  this,  attention  should  be  given  only  to  the  DK  regions  and  the  degrading  effects  of  \R  light  leakage 
through  M  should  be  considered.  For  long  term  dark  storage,  application  of  +Vq  to  (after  writing  with 
+V0  on  E^)  appears  preferable. 


4.  Sensitometry  effects 


4.1  Sensitometry 


Sensitometry  data  is  the  first  information  necessary  for  an  opticallv  addressed  SLM.  In  Figure  3  we  shown 
sensitometry  data  obtained  for  positive  and  negative  writing  on  photo-DKDP.  These  data  were  obtained  using  a 
constant  I\y  -  l8uW/cm^  illumination  from  an  argon  laser  with  \y  ■  514nra  and  a  low  I;y  *  20uW/cto^  readout 
light  intensity.  A  low  Ej  *  0.5uJ/cm^  threshold  was  observed  for  both  cases  with  a  far  lower  exposure  needed 
to  reach  80%  saturation  for  positive  writing  (Efl.g  *  75uJ/cm^)  than  for  negative  writing  (290uJ/cm^).  From 
these  data,  we  also  find  the  dynamic  range  of  the  device  to  be  560/0.5  or  above  1000:1  and  its  contrast  ratio 
to  be  above  100:1.  The  superior  sensitivity  obtained  for  positive  writing  is  due  to  the  better  hole  condi¬ 
tion  in  Se.  In  the  photo-DKDP,  the  image  is  stored  in  the  photo-conductor.  This  is  contrary  to  the  case  of 
operation  of  the  LCLV®  and  other  SLMs.  We  now  discuss  the  Se  photo-conductor  in  some  depth. 

4,2  Quantum  efficiency 

The  quantum  efficiency  n  with  which  photo-carriers  are  generated  in  Se  depends  on  the  field  E  across  the 
Se  according  to^.lO 

n  -  n0  exp[-EQ/kT  +  B^'/kT]  ,  (2) 

where  Hq,  Bq,  and  Eq  are  constants  and  T  is  temperature.  The  initial  field  across  Se  is  approximately  lOOv. 
For  a  lOura  Se  layer,  the  field  is  lO^V/cm-  and  n  =  20%.  However,  our  present  concern  lies  in  the  fact  that 
during  a  long  writing  exposure,  the  voltage  across  the  Se  decreases.  Thus,  photo-electric  conversion  in  Se 
will  be  better  for  the  initial  portion  of  exposure  (when  a  large  E  exists)  then  for  latter  parts  of  the  ex¬ 
posure  (when  E  is  loss). 


■1 
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4.3  Charge  carrier  effects 

Next  we  consider  the  charge  transfer  efficiency  R  in  Se.  This  is  proportional  to 

R  -  r/tR  -  UT(V/d2)  ,  (3) 

where  t  is  the  lifetime  and  t^  is  the  transit  time  of  the  charge  carriers.  For  a  Se  of  thickness  d,  the  drift 
velocity  is  v  *  uV/d  and  t^  *  d^/yV  from  which  (3)  results.  For  d  =  lOum,  t^  *  O.Susec  for  holes  and  a  far 
larger  lOOusec  for  electrons  as  noted  earlier. 

From  (3),  we  see  that  R  increases  with  V  (recall  from  (2)  that  n  also  increased  with  V).  Thus,  we  expect 
better  sensitivity  in  the  photo-DKDP  for  short  exposures  than  for  long  ones.  Moreso,  static  sensitometry 
tests  (such  as  the  ones  presented  in  Figure  2)  using  long  Ty  exposure  times  will  yield  different  results 
(inferior  to  those  obtained  dynamically  with  short  Tw  exposure  times).  Thus,  proper  SLM  test  and  evaluation 
procedures  are  important  because  of  the  voltage  dependent  sensitometry  of  the  Se  photo-conductor  used. 

4.4  ut  product 

For  exposure  times  larger  than  the  transit  times,  charge  q  is  accumulated  at  M  as 

q  -  (N0eVyT/L2) (l-exp(-t) ]  .  (4) 

For  large  Ty,  the  exponential  term  is  zero  and  q  is  proportional  to  the  ut  product.  For  hoLes,  this  is 
1.  3xl0~bcnw  V,  whereas  for  electrons  it  is  four  times  less  or  3. 2xl(Wcm2/V.  We  thus  expect  holes  to  be  more 
efficient  than  electrons  during  writing  and  hence  positive  writing  to  be  better  than  negative  writing.  From 
Figure  3  at  low  Ty  *  0.3sec,  we  find  a  4:1  difference  in  sensitivity  between  positive  and  negative  writing 
(in  excellent  agreement  with  the  predicted  values).  For  longer  exposures,  the  difference  is  less,  thus  indi¬ 
cating  a  field  and  carrier  dependence  for  n  and  the  ut  product. 

4.5  Instituo  image  subtraction  and  addition 

One  of  the  new  applications  of  this  SLM  with  which  we  are  concerned  is  the  subtraction  of  two  frames  of 
data  for  purposes  of  edge  enhancement  preprocessing!-!  and  for  image  registration  for  track  assembly  analy¬ 
sis.  12  The  different  sensitivity  for  hole  and  electron  conduction  and  the  field  dependent  sensitometry  of 
this  SLM  are  of  practical  concern  in  such  interframe  operations.  The  factor  of  four  difference  in  the  ut 
product  of  the  charge  efficiency  has  been  noted  by  others. 9, 10  The  solution  to  this  problem  is  first  the  use 
of  low  exposure  times,  thereby  decreasing  the  time  dependence  of  sensitometry  on  the  time  varying  field  across 
the  Se.  In  Figure  4,  we  show  the  transmittance  vs.  exposure  curves  for  DKDP  for  various  ut  products. 1  These 
curves  indicate  that  by  properly  doping  the  material  to  obtain  ut  products  in  the  range  of  3x10*7-10-5,  ut 
variations  by  even  a  factor  of  10  will  have  little  effect  on  transmittance.  With  the  much  smaller  3xl0”7-10“8 
ut  products  in  our  photo-DKDP  SLM,  small  ut  differences  result  in  large  transmittance  differences  as  seen  in 
the  lower  curves  of  Figure  4. 


5.  Modulation  transfer  function  (MTF) 


5.1  Introduction 


The  resolution  of  any  SLM  is  one  of  the  parameters  of  utmost  concern.  As  we  have  noted :  (1)  unnorma¬ 

lized  MTF  data  is  preferable  to  better  convey  true  device  performance  and  to  better  compare  different  SLMs, 

(2)  interferometric  MTF  data  is  more  reliable  and  preferable,  (3)  linear  MTF  data  is  vital  when  accuracy  is 
of  concern,  (4)  voltage  dependent  sensitometry  effects  must  be  considered  in  properly  selecting  bias  levels 
for  MTF  experiments,  (5)  the  depth-of-focus  of  the  optics  and  associated  addressing  system  used  in  practice 
must  be  compatible  with  those  used  during  the  MTF  data  acquisition  experiments.  In  this  Section,  we  briefly 
describe  these  five  Issues  and  include  our  MTF  data  obtained  on  the  photo-DKDP  together  with  its  comparison 
to  the  theoretically  expected  results. 

5.2  MTF  techniques 


An  MTF  curve  is  a  plot  of  contrast  ratio  vs.  spatial  frequency.  As  noted, ^  the  use  of  unnormalized  MTF 
data  (m  -  m0llt  (u’)  vs.  n *  rather  than  mnonTi  =  mouC  (u'J/mout  (0)  vs.  u’)  is  preferable  for  more  valid  com¬ 
parison  between  SLMs.  A  constant  1002  input  modulation  is  assumed  and  the  exact  test  conditions  must  be 
noted.  Care  should  also  be  taken  to  obtain  at  least  three  approximately  constant  low  u’  mouC  values  before 
ceasing  data  acquisition.  From  a  two  year  program  involving  first-hand  experiments  on  many  SLMs,  we  have 
found  interferometric  MTF  data  to  bo  far  more  reliable  and  useful  than  any  other  form  of  SLM  resolution  mea¬ 
surement.  Because  the  Interferometric  pattern  is  in  focus  over  the  entire  thickness  of  the  SLM,  operator 
errors  arc  greatly  reduced.  In  addition,  by  restricting  the  input  modulation,  monitoring  the  (Tq) *  first 
(I|)  and  second  (It)  order  terms  in  the  Fourier  transform  pattern,  second  harmonic  distortion  can  be  con¬ 
trolled  and  gray  scale  and  linear  MTF  data  can  be  obtained.  As  noted  in  Section  4,  when  voltage  dependent 
sensitometry  effects  are  present,  the  sensitometry  and  MTF  experiments  must  be  conducted  similarly  to  properly 
determine  the  corresponding  bias  level  to  be  used.  tn  Sect.  7,  the  new  Bessel  transfer  function  derived  for 
this  SLM  indicates  the  origin  of  the  harmonic  distortion. 


;}  .  . 
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5.3  Depth-of-£ ocus  effects 

The  first  major  shortcoming  of  intert erometr ical lv  obtained  MTF  data  is  its  applicability  to  different  SLM 
uses,  such  as  when  the  input  data  is  scanned  or  imaged  onto  the  SLM  in  practice.  Specifically,  the  lens  sys¬ 
tem  and  data  recording  technique  actually  used  must  have  the  depth-of-focus  and  focusing  accuracy  of  the  in- 
cerferometric  recording  method  if  the  results  of  the  MTF  experiment  are  to  be  of  use.  Interferometric  MTF 
data  yields  the  largest  possible  SLM  resolution;  whether  this  resolution  and  performance  are  obtainable  in 
practice  depends  upon  the  optical  system  and  addressing  method  employed. 

If  the  photo-conductor  in  the  SLM  is  J  um  thick,  the  associated  imaging  lens  must  have  a  depth-of-focus 
above  d  um.  The  spatial  frequency  limit  for  such  a  svstem  islb  fi/o^L  cy/mm.  Conversely,  if  the  depth-of- 
focus  of  the  lens  used  is  d^,  then  dt  should  be  less  than  d  um  and  MTF  data  beyond  the  same  limit  noted 

above  should  not  be  used.  This  brie?  discussion  assumes  that  the  photo-conductor  is  active  throughout  its 
entire  volume  and  thickness  and  Ignores  photo-corductor  response  variations  normal  to  the  optical  axis.  The 
depth-of-focus  of  a  lens  of  focal  length  f^  and  apertute  D  used  at  a  wavelength  X  is 

SfL  -  8X(fL/D)2  .  (5) 

The  uncertainty  in  the  location  of  the  focus  in  the  plane  normal  to  the  optical  axis  is 

«y  -  (X/4)(fL/D)  .  (6) 

The  corresponding  spatial  frequency  at  which  problems  will  be  encountered  in  the  use  of  such  a  lens  is 

v  -  l/(2«y>  .  (7) 

max 

With  properly  selected  lenses  used  in  the  actual  recording  system,  these  problems  can  be  avoided.  For  the 
photo-DKDP,  lenses  with  f./D  *  frf  *1.-  allow  us  to  reach  a  spatial  frequenev  limit  of  vmax  *  2200cv/mm.  The 
above  analysis  has  assumed  diffraction  limited  systems  and  clearly  other  system  aberrations  will  dominate  such 
a  process  before  this  limit  is  reached. 

5.4  MTF  data 


Interferometric  MTF  data  were  obtained  on  the  photo-DKDP  with  a  bias  exposure  point  in  the  central  linear 
region  of  the  sensitometry  curve  chosen  (see  Figure  3).  Input  modulation  was  increased  while  monitoring  Iq, 
1^,  and  I2.  I2  was  kept  20dB  below  1^,  thus  insuring  less  then  IT  interharmonic  distortion.  The  resultant 
Il/Io  measured  values  vs.  u’  were  converted  to  output  amplitude  modulation  usingl^ 


m 

out 


1  +  4^/1 


(8) 


The  MTF  curve  of  Figure  5  results.  From  it,  we  find  a  maximum  modulation  of  90T  and  a  resolution  at  50T  of 
the  peak  modulation  of  16cv/mm.  Note  that  this  MTF  data  includes  gray  scale  and  linear  recording  issues  and 
applies  for  less  chan  IZ  distortion. 

5.5  Comparison  to  theory 

Roachl?  and  ochers!8,19  have  analyzed  Che  resolution  of  such  SLMs.  They  have  solved  the  Laplace  Equation 
in  the  DKDP,  M,  and  Se  regions  subject  to  the  correct  boundary  conditions  and  assumed  a  surface  charge  density 
o  of  unity  and  spatial  frequency  v 


a  -  Op  cosivvx  . 

They  found  Che  resultant  voltage  across  Che  DKDP  to  be 

V  «  V^fv)  cosEmvx  +  8ias 


(9) 

(10) 


where 


V2(v) 


on/2vvc0 


(ez/ex)  1  Coth[  (ex/ey)  -ttva)  +  coth(2wvl>) 


(11) 


in  which  a  is  the  thickness  of  the  DKDP  and  b  is  the  thickness  of  the  mirror.  From  (11),  we  see  that 
V2(v)/V2<0)  decreases  as  v  Increases.  A  plot  of  ^2^0  vs*  v  is  amplitude  modulation  or  MTF.  The  corre¬ 
sponding  theoretical  curve  is  shown  in  Figure  6.  It  is  seen  to  be  in  excellent  agreement  with  the  experimen¬ 
tal  curve  obtained  in  Figure  5. 
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6.  Other  phot  >-PKDP  tests 


6.1  Introduction 


In  the  following  Subsections,  we  describe  several  additional  tests  performed  on  the  photo-DKDP  SLM. 

6.2  Dark  storage 

The  dark  storage  of  the  photo-DKDP  (Ixr  *  0)  Is  determined  by  the  time  constants  of  the  Se  and  the  DKDpl»20 

xSe  *  53sec,  tDKJ3p  =  2671sec  .  (12) 

Using  an  RC  model  for  the  photo-DKDP,  we  find  an  expected  dark.  DK  time  constant  of  Since  intensity  is 

measured  in  all  experiments  and  since  I*V^  at  low  V,  the  measured  time  constant  is  expected  to  be  r/2  *  1335 
sec.  Experimental  data  obtained  by  recording  a  5cy/mm  squarewave  pattern  on  the  DKDP  and  measuring  the  Iq 
and  Ii  Fourier  components  vs.  time  confirmed  the  theoretical  predictions.  Iq  and  1^  were  found  to  decay  at 
the  same  rate  indicating  that  dark  storage  decay  results  in  a  loss  of  charge  and  hence  output  light  without  a 
loss  of  definition  or  modulation  (I^/Iq).  The  measured  dark  decay  time  constant  was  x  *  1500sec,  in  good 
agreement  with  the  predicted  value  of  1335sec.  For  longer  storage  times,  an  alternate  model  is  necessary. 

6.3  Readout  decay 


Under  readout  (I^«  4  0)  Che  stored  pattern  is  expected  to  decay  faster  and  differently  than  under  dark 
storage.  Using  I^r  -  50uW/cm^,  the  decay  time  constants  for  the  zero-order  Tq  and  first-order  Fourier 
terms  were  measured  to  be 


Tq  3  1400sec,  x^  3  llOOsec  .  (13) 

These  values  are  less  than  the  dark  storage  decay  time  constants  as  expected,  but  are  still  quite  long.  Since 
T1  <  T0»  we  sce  chat  t^e  modulation  (li/Io)  of  the  recorded  data  decreases  under  readout  (only  by  about  10%) , 
whereas  it  remained  constant  under  dark  storage.  Thus,  we  conclude  that  no  appreciable  readout  error  effects 
will  arise  and  that  long  15min  times  are  acceptable  with  this  SLM. 

6.4  Erase  time 

Erase  time  in  our  experiments  were  much  longer  (4sec  using  a  broadband  8raW/cnr  arc  lamp  A-  source).  This 
was  due  to  the  voltage  dependent  sensitomecry  noted  earlier,  to  the  low  I^£  intensity  available  and  to  the 
broadband  nature  of  the  AE  source  used.  Erase  times  of  1msec  are  possible  with  alternate  techniques  as  we 
describe  in  Section  7. 

6.5  Optical  path  difference 

The  optical  quality  of  an  SLM  is  of  concern  in  coherent  optical  data  processing  applications.  The  wedge 
present  in  the  DKDP,  the  non-uniform  temperature  of  the  target,  and  the  optical  quality  of  its  surface  Limited 
the  rms  OPD  phase  quality  of  the  photo-DKDP  unit  we  tested.  Over  its  central  2x2cm^  area,  the  phase  error 
was  A/4.  Interferometric  tests  are  usually  used  to  obtain  such  data,  although  point-by-point  OPD  information 
is  more  useful  in  estimating  the  effects  of  such  OPD  errors  on  system  performance. 21 *22 

6.6  Scatter  level 


The  scatter  level  of  an  SLM  is  another  factor  that  limits  the  processing  gain  obtainable  using^it  and  its 
ability  to  process  signals  with  low  SNR.  To  obtain  scatter  level  data  on  the  photo-DKDP,  a  7x7mm-  input  aper¬ 
ture  was  used.  The  Fourier  transform  pattern  for  this  input  was  scanned  with  all  components  in  place  (except 
the  photo-DKDP  itself)  and  a  scatter  level  or  optical  system  maximum  dynamic  range  of  52dB  was  measured.  The 
same  experiment  was  repeated  with  the  input  aperture  imaged  onto  the  photo-DKDP.  The  observed  scatter  level 
was  50dB.  Thus,  use  of  the  photo-DKDP  in  coherent  optical  data  processing  systems  appears  to  be  most  excel¬ 
lent  and  should  Introduce  less  than  2dB  of  noise. 

7.  Now  formulations  of  spatial  birefringence  and  operation 


7.1  Introduction 


G.  Lebreton  and  E.  de  Bazelaire  at  Gessy  Laboratories  of  the  University  of  Toulon  have  considered  the 
operation  of  the  photo-DKDP  SLM  in  depth.  A  new  transfer  function  for  the  device  with  attention  to  its 
polarization  effects  has  been  produced  (Section  7.2),  and  has  resulted  in  an  improved  response  linearity 
(Section  7.3)  and  was  experimentally  verified  (Section  7.i).  The  diffracted  light  from 
such  a  device  was  shown  to  be  of  constant  polarization  independent  of  the  input  modulation  or  spatial  fre¬ 
quency.  This  formulation  also  resulted  in  new  write  and  read  methods  (Section  7.6)  with  the  ability  to  sup¬ 
press  the  input  bias  level  (Section  7.5)  and  achieve  faster  (- lmsec ) erase  times.  The  intent  and  application 
of  this  device  being  considered  in  France  is  as  the  real-time  input  transducer  for  radar  and  sonar  signal 
processing  applications. 
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The  classic  approach^ to  the  analysis  of  the  spati  »1  birefringent:  modulation  of  the  photo-DKDP  uses 
particular  cases  of  the  well-known  amplitude  and  phase  modulation.  However,  a  complex  analysis-^  involving 
an  extensive  study  of  the  effects  of  polarized  light  in  this  SLM  and  experiments  performed  using  an  ellipso- 
meter  provided  several  new  results  which  indicate  that  modifications  to  the  traditional  description  of  the 
spatial  birefringent  modulation  in  the  photo-DKDP  and  other  SLMs  are  in  order.  These  results  are  summarized 
below  and  discussed  in  subsequent  Subsections. 

(1)  In  a  Poincare  sphere  descript  ion  ,24  the  diffracted  light  is  a  symmetric  version  of  the  undiffracted 
light  with  respect  Co  the  birefringent  axis  of  the  modulating  crystal.  This  theory^  was  verified  by  experi¬ 
ment.  26  a  particular  case  of  the  above  theorem  follows  in  (2). 

(2)  When  the  Poincare  sphere  representation  of  the  incident  light  lies  on  the  circle  normal  to  the  crys¬ 
tal's  birefringent  axis,  the  polarization  of  the  diffracted  light  is  orthogonal  to  that  of  the  input  light. 27 
The  conclusion  is  that  when  properly  oriented  linearly  polarized  incident  light  is  used,  the  polarization  of 
the  diffracted  light  will  be  linear  and  orthogonal  to  the  polarization  of  the  incident  light  and  to  the  polar¬ 
ization  of  the  zero-order  diffracted  light.  This  effect  has  also  been  observed  in  shear  mode  acousto-optic 
SL!Is28  and  in  BSO.24 


(3)  The  polarization  of  the  diffracted  light  noted  in  (2)  is  independent  of  the  amplitude  and  spatial  fre¬ 
quency  of  the  input  modulating  signal.  Only  the  angle  of  diffraction  depends  on  the  input  spatial  frequency. 

(4)  The  amplitude  of  the  diffracted  light  is  described  by  a  Bessel  function  (as  in  conventional  phase 
modulation),  and  not  by  che  classic  crossed  polarizer  modulation  expression. 

(5)  This  leads  to  a  new  transfer  function  with  improved  output  linearity  and  to  new  write  and  read  modes 
of  operation  for  the  SLM. 


7.2  New  spatial  birefringence  transfer  function 


We  can  view  the  incident  readout  light  to  be  propagating  in  the  z  direction  with  equal  electric  field  vec¬ 
tors  in  the  (x,y)  plane.  Elliptical  polarization  can  be  described  by  linearly  polarized  orthogonal  components 
Ax  and  Ay  on  the  fast  and  slow  electro-optic  axes  of  the  crystal. 


A;(x,y)  -  Ax  exp(jtx(x,y) ) 


(14a) 


Ay(x.y)  *  Ay  exp(j*y(x,y) ]  .  (14b) 

Since  any  spatial  modulated  signal  can  be  represented  (along  x  and  y)  by  a  Fourier  integration  3nd  since  opti- 
cal  Fourier  transform  systems  are  linear,  we  restrict  attention  to  one  sinusoidal  component  of  the  signal  with 
amplitude  a,  bias  level  V'q  and  spatial  frequency  fQ.  The  voltage  across  the  DKDP  for  such  a  signal  is 

V(x)  *  (a/ 2 ) VQ  sin(2Trf0x)  (15) 

and  the  corresponding  phase  modulation  terms  in  (14)  are 

♦v  "  *  (WX)nQr63(a/2)  VQ  sin(2TffQx)  *  (<J>/2)  sin(2rf0x)  .  (16) 

The  A'  terras  in  (14)  can  then  be  expressed  as  a  series  of  Bessel  functions  J  of  the  first  kind.  Since 
Jq  is  an  even  function  and  all  other  Jn  are  odd,  the  total  diffraction  pattern  (after  filtering  of  harmonic 
frequencies)  can  be  described  by 

(A'U^)]  -  (A*+Ay)  {J0(‘>/2)  6( fx)  +  (^-Ay)  (J1fV2)  («S(fx-fQ)  +  I  (17) 

From  (17)  we  see  that:  (1)  the  polarization  of  the  zero-order  diffracted  pattern  (first  term)  is  the  same  as 
the  polarization  of  the  incident  light.  We  also  see  that:  (2)  the  polarization  of  the  first-order  diffracted 
or  modulated  light  (second  terra)  has  a  polarization  that  is  symmetric  with  respect  to  the  poLarization  of  the 
Incident  light  with  respect  to  the  birefringent  axis  in  a  Poincare  sphere  description  of  the  process,  (3)  the 
Input  spatial  frequency  affects  only  che  angle  of  diffraction,  and  that  (4)  che  amplitude  of  the  diffracted 
light  is  related  to  the  input  by  Bessel  functions  as  in  the  classic  phase  modulation  case. 


7.3  Linearity  of  response 


From  (17),  we  see  that  the  amplitude  of  the  diffracted  light  is  related  lo  the  phase  by  Bessel  functions 
Jn  f*/2)  rather  than  by  the  conventional  sin(J>/2)  equation.  This  new  transfer  function  implies  that  the  actual 
linearity  of  the  device  is  far  better  than  one  would  expect  using  classic  relationships.  At  linearity  error 
for  the  two  transfer  functions  is  summarized  in  Table  2  (the  phase  value  noted  is  the  total  delav  between 
both  components  of  the  input  wave). 


Table  2.  Linearity  of  >  for  sin»,J>/2)  and  Jn(j>/2)  Modulation 


» 

it/3 

2n/4 

tr/2 

2ff/3 

IT 

%  linearity  in 

17. 

27. 

3% 

17. 

22% 

J^($/2)  Modulation 

Sin($/2)  Modulation 

2% 

uz 

3% 

18% 

57% 

7.4  Ellipsoraeter  experiments 

From  Table  2,  we  see  that  the  new  transfer  function  indicates  far  better  linearity  in  response  of  this  SLM 
than  did  prior  classic  formulations.  Such  excellent  linearity  is  vital  in  the  use  of  this  SLM  in  signal  pro¬ 
cessing  applications.  Such  differences  are  not  usually  observed  experimentally  with  small  angle  modulations 
since  they  appear  only  in  the  third-order  of  a  Taylor  series  expansion  of  the  2  functions.  In  the  normal 
experiments  performed  with  cross  polarizers,  the  input  polarization  is  not  always  linear  and  only  the  projec¬ 
tions  of  the  modulated  elliptically  polarized  output  signal  on  two  orthogonal  axes  can  be  measured.  Since 
many  elliptical  polarization  outputs  can  yield  similar  projection  data,  accurate  analysis  of  the  modulation 
obtained  is  not  possible.  To  experimentally  verify  the  remarks  advanced  in  Section  7.2,  an  ellipsometer  was 
constructed  using  a  rotating  \l 2  plate  and  a  fixed  linear  polarizer.  This  allowed  the  ellipticitv  and  orien¬ 
tation  of  the  axes  of  the  eLlipse  to  be  accurately  determined.  The  variable  birefringence  of  the  photo-DKDP 
was  measured  without  spatial  input  modulated  signals  for  different  bias  voltages  from  +160  volts  with  a 
saturated  input  illurainat ion  level.  The  time  of  exposure  was  also  varied  for  +150  volt  bias  levels  with  con¬ 
stant  power  used.  From  di:  ?ct  measurements  with  circularly  polarized  input  light,  the  exact  angle 

in  the  Poincare  sphere  model  was  leterrained.  More  accurate  measurements  were  then  performed  with  the  input 
light  Linearly  polarized  at  45'*  to  the  crvstal’s  biref ringent  axes.  From  these  data,  the  ellipticitv  was  de¬ 
termined.  No  intermediate  optical  system  was  used  to  avoid  parasitic  birefringence  effects. 

The  performance  of  the  photo-DKDP  with  spatially  modulated  input  light  is  quite  different  and  is  not  di¬ 
rectly  describable  using  the  same  Poincare  sphere  description.  Tests  performed  with  a  100  -m  period  Ronchi 
ruling  and  vertically  polarized  readout  light  showed  the  polarization  of  the  zero-order  to  be  vertical  and 
that  of  the  higher  order  terms  to  be  horizontal  (independent  of  the  depth-cf-modulat ion  of  the  input).  This 
verifies  the  theory  advanced  in  Section  7.2.  In  the  above  experiments,  the  bias  level  of  the  input  signal 
was  cancelled  as  explained  in  Sections  7.5  and  7.6.  When  this  was  not  done,  the  polarization  of  the  zero- 
order  spot  was  elliptical  and  related  to  the  depth-of-modulation  of  the  input  signal  by  the  Poincare  sphere 
model. 

7.5  Input  bias  level  suppression 

Several  authors-9  have  noted  that  a  suppression  of  the  zero-order  diffracted  light  can  be  obtained  by 
illuminating  the  SLM  with  a  uniform  light  distribution  equal  to  the  average  amplitude  of  the  image  with  the 
opposite  polarity  Vq  applied  to  the  crystal.  Since  the  zero-order  diffracted  light  is  described  by 

A0  -  o(fx)[Ax  exp(-j<fr/2)  +  Ay  exp(j$/2)]  ,  (18) 

and  since  the  polarization  of  this  term  is  different  from  that  of  the  diffracted  light  (Section  7.2),  it  can¬ 
not  be  fully  cancelled  (using  cross-polarizers)  by  the  indicated  second  uniform  exposure. 

7.6  New  write,  erase,  and  bias  level  suppression  operation 

New  erase  and  write  modes  of  operation  are  necessary  to  achieve  optimum  performance  in  this  SLM.  These  are 
suggested  by  the  new  polarization  formulation  advanced  in  Section  7.2.  All  results  have  beer,  experimentally 
verified.  The  technique  used  is  to  write  the  original  image  with  +Vq  applied  to  E|  as  before.  During  era¬ 
sure,  the  applied  voltage  to  E^  is  reversed  to  -Vq  and  a  strong  erase  photo-flash  is  applied.  This  causes  the 
voltage  at  M  due  to  BR  and  DK  areas  of  the  crystal  to  both  saturate  to  the  same  large  negative  value.  Of 
concern  and  note  is  the  fact  that  both  the  DK  and  BR  crystal  regions  now  have  the  same  voltage  value  at  M  and 
that  this  value  Is  a  constant  for  all  cases  and  independent  of  the  image  used.  When  a  aew  image  is  recorded 
with  +Vq  applied,  an  input  bias  illumination  lev*. 1  equal  to  the  constant  saturation  level  present  is  used. 
This  results  in  the  recording  of  a  zero  mean  pattern  on  the  photo-DKDP  and  full  use  of  the  phase  modulation 
theory  described  in  Section  7.2.  This  new  write  and  erase  procedure  avoids  the  multi-step  process  previously 
employed,  fully  cancels  the  bias  level  of  the  input  data,  and  allows  faster  erase  time  and  cycle  tines  for 
this  SLM  and  results  in  the  full  use  of  the  Bessel  phase  function  modulation  features  noted  in  Section  7,2 
with  their  associated  other  advantages. 


fl.  AppHi •  at  ions 


8.1  Introduction 

The  purpose  of  any  SLM  is  to  serve  as  the  real-time  and  reusable  transducer  for  diverse  optical  computing 
applications.  In  the  following  Subsect  ions  we  briefly  summarize  several  of  the  new  applications  to  which  wc 
have  applied  the  photo-DKDP  SLM.  These  data  are  more  fully  described  in  (14).  They  are  of  particular 
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interest  because  of  the  comparison  between  theory  and  experiment  and  between  the  use  of  film  inputs  and 
real-time  SLMs. 

8.2  Edge  enhancement  preprocessing  and  image  registration  for  track  assembly 

Edge  enhancement  preprocessing  is  a  useful  step  in  the  correlation  of  multi-senaor  imagery. H  By  ex¬ 
posing  the  photo-DKDP  to  two  successive  patterns  with  different  polarities  of  applied  voltage  present  between 
each  exposure,  the  subtraction  of  the  two  2-D  images  results.  This  is  also  of  use  in  image  registration  for 
track  assembly. 12  If  the  two  images  are  defocused  versions  of  the  same  imaee,  the  resultant  difference  image 
is  an  edge-enhanced  version  of  the  original  one.  This  operation  is  of  use  in  preprocessing  for  multi-sensor 
image  pattern  recognition.  Initial  tests  on  the  use  of  the  photo-DKDP  SUM  for  both  of  these  operations^ 
have  been  most  promising. 

8.3  Optical  pattern  recognition 

The  input,  reference  and  output  correlation  patterns  obtained  using  the  photo-DKDP  SUM  as  the  input  plane 
transducer  in  a  conventional  matched  spatial  filter  frequency  plane  correlator  are  shown  in  Figure  7.  The 
location  of  the  output  correlation  peak  is  seen  to  be  proportional  to  the  location  of  the  reference  function 
within  the  input  scene.  We  have  experimented  with  the  use  of  this  technique  and  the  photo-DKDP  SLM  in  missile 
guidance  and  optical  word  recognition  (OWR)  applications.  An  OWR  experiment  in  which  there  were  4  occurences 
of  the  reference  word  was  repeated  using  inputs  on  both  film  and  on  the  photo-DKDP.  The  resultant  Ip  values 
for  the  four  output  correlation  peaks  were  30,  28,  28,  27  when  the  photo-DKDP  was  used  and  were  remarkably 
similar,  37,  34,  32,  29  when  film  inputs  were  used.  The  widths  of  the  output  correlation  peaks  fn  this  OWR 
example  were  also  compared.  The  theoretical  width  was  computed  to  be  63  um.  The  width  obtained  using  the 
photo-DKDP  inputs  was  90  um,  whereas  60  um  widths  resulted  when  film  inputs  were  used.  The  low  modulation 
of  the  photo-DKDP  at  the  stroke  width  of  the  characters  used  appears  to  be  the  source  of  the  larger  correla¬ 
tion  peak  widths  obtained  using  the  real-time  transducer. 

8.4  Optical  Signal  Processing 

The  output  correlation  pattern  obtained  using  photo-DKDP  tn  a  crossed  input  ambiguity  function  proces¬ 
sor  is  shown  in  Figure  3  lor  a  simple  double-pulse  signal.  The  width  in  doppler  of  the  optically  pro¬ 

duced  outputs  were  seen  to  be  0.173  mm  (versus  a  theoretical  0.17  mm  value).  The  width  in  range  of  the  output 
was  7.25  mm  (versus  a  theoretical  7.20  (ran  value).  The  peak  to  side  lobe  ratio  measured  was  4.3  (versus  a 
theoretical  value  of  4).  These  experimental  results  were  thus  in  excellent  agreement  with  those  predicted  by 
theorv.  Similar  excellent  performance  was  obtained  when  the  plu>co-DKDP  was  used  as  the  real-time  SLM  in  a 
passive  ambiguity  surface  detection  application 14  and  in  correlations  of  active  sonar  signals. 
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Figure  1.  Schematic  diagram  of  the  phoco-DKDP 
spacial  light  modulator. 
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Figure  3.  Sensitometrv  data  for  the  photo-DKDP  SLM. 

(a)  Positive  writing  with  +Vq  applied  to 
Ei  during  writing; 

(b)  Negative  writing  with  -VQ  applied  to 
during  writing. 


Figure  4.  Transmission  versus  exposure  for  the 

photo-DKDP  SLM  for  different  pi  carrier 
products. * 


Figure  2.  Voltage  versus  distance  distribution  in 
the  photo-DKDP  SLM  at  different  stops  of 
operation  and  for  a  bright  (BR)  and  dark 
(DK)  (illuminated  and  unilluminated)  regions. 
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Figure  5.  Experimental  nonlinear  MTF  (unnormalized) 
for  the  photo-DKDP  SLM  including  gray 
scale  and  with  less  than  1  percent 
harmonic  distortion. 


Figure  6.  Theoretical  MTF  curve  for  the  photo-DKDP 
SLM. 


(b) 


Figure  7.  Real-time  optical  pattern  recognition  using 
the  photo-DKDP  SLM,  (a)  Input;  (b)  Ref¬ 
erence;  (c)  Output  correlation  pattern. 


Figure  8.  Real-time  optical  signal  processing  using 
the  ph<Hn-PKDP  SIM  as  the  Input  transducer 
for  a  crossed  input  ambiguity  function  pro- 
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Equalizing  and  coherence  measure  correlators 

David  Casasent  and  Alan  Furman 


A  correlator  is  described  that  combines  optical  and  electronic  processing.  The  optical  section  is  the  first 
Fourier  transform  taking  stage  of  a  joint  transform  correlator;  its  output  is  electronically  processed  by  a 
modified  spectrum  analyzer.  Increased  flexibility  and  various  processing  operations  beyond  those  normally 
possible  in  an  optical  system  result. 


I.  Introduction 

The  importance  of  the  correlation  operation  in  image 
and  signal  processing  is  well  accepted.  Many  diverse 
optical  correlator  systems  have  been  devised,1-7  be¬ 
ginning  with  the  holographic  matched  filter  correla¬ 
tor,1-2  that  enable  this  correlation  operation  to  be  per¬ 
formed  in  parallel  on  2-D  imagery  or  multichannel  1-D 
data  and  at  high  throughput  rates.  The  basic  optical 
system  which  we  chose  to  modify  is  the  joint  transform 
correlator  (JTC).3-5  Rau6  had  previously  suggested 
detection  of  the  joint  transform  of  two  functions  (placed 
side  by  side  in  the  input  plane)  by  a  vidicon  camera.  A 
spectrum  analyzer  was  applied  to  the  camera’s  video 
signal,  yielding  the  correlation  of  the  two  inputs  in  a 
demonstration  of  optical  character  recognition.  In 
temporal  offset  correlation,  Macovski  and  Ramsey7 
devised  a  method  whereby  a  video  signal  equivalent  to 
the  above  one  was  produced  by  transforming  the  co¬ 
herent  superposition  of  the  two  images  (fully  overlap¬ 
ping)  after  having  frequency-shifted  the  light  used  to 
illuminate  one  of  the  images.  The  transform  plane 
intensity  then  contains  a  temporal  ac  term  composed 
of  a  carrier  (at  the  shift  frequency)  modulated  by  the 
complex  transform  product.  Since  the  width  of  the 
input  format  is  reduced  typically  by  a  factor  of  3,  the 
transform  plane  detector  spatial  resolution  required  is 
reduced  proportionally;  however,  because  an  ac  (in 
time)  intensity  distribution  must  be  detected,  a  non¬ 
integrating  (e.g.,  image  dissector)  detector  must  be 
used. 

In  the  system  to  be  described  herein,  the  joint 
transform  detection  system  is  modified,  and  an  equal¬ 
izing  correlator  produced  that  enables  compensation 
of  various  types  of  signal  distortions  to  be  realized.  A 
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theoretical  description  (Sec.  II)  of  the  electronic  readout 
joint  transform  correlator  and  experimental  demon¬ 
strations  of  it  (Sec.  Ill)  are  followed  by  similar  sections 
for  the  equalizing  correlator  (Secs.  IV  and  V).  We 
conclude  with  a  discussion  of  other  extensions  of  this 
system  including  a  coherence  measure  operation  (Sec. 
VI). 

II.  Electronic  JTC  Readout;  Theoretical 

The  conventional  joint  transform  correlator3  5  is  re¬ 
viewed  first.  In  this  system,  the  two  functions  f  and  g 
to  be  correlated,  both  assumed  to  have  a  width  of  6,  are 
placed  side  by  side  in  the  input  plane  P0  with  a  center- 
to-center  spacing  26. 8  We  represent  the  transmittance 
of  PQ  by 

t,-(x)  *  fix  +  b)  +g(x  -  b).  (1) 

We  use  1-D  notation  both  for  simplicity  and  because 
we  will  deal  mainly  with  1-D  processing  in  this  paper. 
The  Fourier  transform  plane  intensity  distribution 
(modulus  squared  of  the  transform  of  t, )  is 

/i(u)  =  |F(u)|2  +  |G(u)|z  +  2|fj  |(»|  cos|4xu6  +  <2>(u )|,  (2) 

where  F  and  G  are  the  Fourier  transforms  of  f  and  g, 
and  the  coordinate  x  i  of  the  transform  plane  P\  is  re¬ 
lated  to  the  spatial  frequency  u  by  u  =  x\/\fi,.  The 
focal  length  of  the  transform  lens  is  //.,  and  the  phase 
term  <t>(u)  =  arg  G(u )  —  arg  F(u )  equals  the  difference 
between  the  phases  of  the  transforms  of  the  two  func¬ 
tions. 

This  Fourier  transform  distribution  is  recorded  on 
an  optically  addressed  spatial  light  modulator  (SLM),9 
and  the  SLM  is  now  read  by  illuminating  it  with  a  plane 
wave.  Assuming  that  the  SLM’s  amplitude  transmit¬ 
tance  is  approximately  a  linear  function  of  writing  in¬ 
tensity  /|,  we  obtain  at  the  output  plane  P 2,  upon 
forming  the  Fourier  transform  of  this  transmittance. 

u(x2)  =  /©/  +  g®K  +  /©«  •  *  (X-2+  2fc)  +#?©/  •  i  <*Z  -  26). 

(3) 

where  ©denotes  correlation,  and  *  denotes  convolution. 
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Fig.  2.  Block  diagram  of  the  electronic  section  of  the  basic  electronic 
JTC  system. 


This  is  the  desired  correlation  of  the  input  functions 
centered  at  xi  =  ±26  (assuming  equal  focal  lengths  for 
both  transform  lenses). 

In  the  electronic  readout  JTC  system,  the  intensity 
distribution  in  Eq.  (2)  is  converted  to  an  electrical  time 
function  (video  signal)  by  a  scanning  photodetector  as 
shown  in  Fig.  1,  and  the  second  transform  is  produced 
in  modified  form  electronically  as  shown  in  Fig.  2.  We 
represent  the  output  electronic  signal  from  the  detector 
at  P]  in  1-D  by 


v(t)  =  R/,(u).  (4) 

where  R  is  the  detector’s  responsivity  in  volts/unit  in¬ 
tensity,  s  is  the  detector’s  scanning  speed,  and  hence  x  i 
=  st,  and  u  =  st/Xfi This  video  signal  v(t)  is  multi¬ 
plied  by  quadrature  sinusoids  sin(uof)  and  cos(u)of), 
these  two  separate  products  integrated,  and  the  inte¬ 
grals  squared  and  summed.  We  then  obtain  a  new 
output: 


i>'(£) 


“  (^7^)  \L  ^(u)exp02iru{)du| 
“  i~T^)  rect(u/2u0))|2 


u(i)  smvotdt 


2 


r 


=  |sinc(2uof)  *  l/©/  +  +  S©/ *  *(£-  26) 

+  /©g*S<£  +  26)]|2,  (5) 


where  T  is  the  time  constant  of  the  integrator  (the 
product  of  a  resistance  and  capacitance  in  the  typical 
op  amp  integrator),  2fo  is  the  line  scan  time,  u o  =  s£o/ 
Xfi,,  and  {  =  u>()XfiJ2ws.  The  sine  function  in  Eq.  (5) 
represents  the  finite  frequency  bandwidth  of  the  cor¬ 
relator  system.  If  the  transform  plane  P i  is  repetitively 
scanned,  the  integrators  in  Fig.  2  are  discharged  to  zero 


initial  value  after  each  line,  and  the  oscillator’s  fre¬ 
quency  wo  (and  hence  correlation  shift  parameter  £)  are 
incremented  between  scans,  the  desired  output  corre¬ 
lation  of  the  two  input  functions  results. 


A.  Anatomy  of  the  Joint  Transform  Pattern 

We  now  examine  the  joint  transform  intensity 
waveform  at  P\  of  Fig.  1  as  the  input  functions  (  and  g 
are  changed  as  an  aid  in  visualizing  the  examples  and 
system  extensions  to  follow.  From  Eq.  (2),  we  see  that 
|F|2  +  |G|2  is  the  slowly  varying  part  of  the  output 
video  signal.  The  important  part  of  this  signal  is  a  co¬ 
sine  function  at  a  frequency  26,  phase  modulated  by  the 
difference  between  the  phases  of  the  transforms  of  the 
two  functions  and  its  strength  proportional  to  the 
product  of  their  magnitudes. 

We  assume  approximately  equal  power  spectra  |F| 
~  |  G  |  in  all  cases  to  simplify  the  analysis.  For  the  case 
of  equal  inputs  f  =  g,  the  expected  output  is  the  auto¬ 
correlation,  <p  =  0,  and  Eq.  (2)  becomes 

/i(u)  =  2|C|2(1  +  cos4iru6).  (6) 

The  recorded  fringe  pattern  or  cos(  J  term  thus  has 
100%  amplitude  modulation.  If  one  input  function  is 
displaced  away  from  the  other  in  Pa,  i.e.,  the  separation 
between /  and  g  is  increased  (say  to  2.56),  then  the  fre¬ 
quency  of  the  fringes  increases  to  2.56.  With  f  ~g,  Eq. 
(2)  now  becomes 

lt(u)  *  2|G|2(1  +  cos5»u6)  =  2|G|2[1  +  cos(4tu6  +  iru6)|.  (7) 

From  the  second  formulation  in  Eq.  (7),  we  see  that  we 
can  also  view  this  as  the  addition  of  a  linear  phase  term 
whose  slope,  or  derivative  with  respect  to  u ,  is  propor¬ 
tional  to  the  shift  in  the  location  of  g  in  P0  and  thus 
contains  data  on  the  location  of  g  in  f. 

If  {  9^  g,  the  fringe  pattern  is  no  longer  a  pure  cosine. 
Equation  (2)  describes  the  resultant  electronic  output 
from  the  plane  P\  detector;  phase  modulation  <p(u)  ^ 
0.  The  power  spectral  density  at  the  carrier  frequency 
26  is  now  decreased.  This  spectral  component  is  the 
correlation,  and  hence  the  loss  in  this  component  due 
to  0  ^  0  causes  an  associated  loss  in  signal-to-noise  ratio 
(SNR)  of  the  correlation  output. 

If  fix)  =  ag(x),  f  and  g  are  identical  except  for  a  de¬ 
creased  strength  for  /  by  a  factor  a.  The  detected 
pattern  is  now 

/](u)  =  | G 1 2(  1  +  o  +  2a  cos4iru6).  (8) 

As  seen,  the  modulation  is  reduced  with  the  amplitude 
of  the  information-bearing  ac  term  reduced  propor¬ 
tional  to  the  decreased  strength  of  /. 

By  electronically  introducing  phase  modulation  into 
v(t),  one  can  compensate  for  phase  differences  between 
F  and  G  such  as  those  due  to  signal  propagation  dif¬ 
ferences  and  P\  detector  scanning  errors.  By  sensing 
the  level  of  the  ac  signal  in  Eq.  (8),  the  presence  of  dif¬ 
ferences  in  the  strengths  of  the  input  functions  can  be 
found  and  compensated  for.  We  discuss  such  exten 
sions  of  this  electronic  readout  JTC  system  later  and  for 
now  only  note  that  considerable  information  on  the 
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Fig.  3.  Example  of  pulse-coded  waveform  correlation:  (a)  input 
transparency;  (b)  correlation  output. 


similarity  as  well  as  the  relative  positions  and  strengths 
of  the  two  input  functions  may  be  extracted  from  the 
amplitude  and  phase  of  the  ac  fringe  signal. 

III.  Autocorrelation  Experiments 

The  optical  system  of  Fig.  1  was  used.  A  50-mW 
He-Ne  laser  source  at  X  =  633  nm  and  equal  focal 
lengths  f  ls  =  /lc  =  381  mm  for  the  spherical  and  cy¬ 
lindrical  lenses  provide  unity  vertical  image  magnifi¬ 
cation  (fis/fic  =  1)  from  Po  to  Pi  with  the  horizontal 
transform  of  the  input  pattern  on  each  line  appearing 
at  the  corresponding  horizontal  output  line  (with  an 
effective  lens  focal  length  h  =  381  mm).  A  linear 
self-scanned  CCD-addressed  photodiode  array  (Reticon 
CCPD-1728)  with  1728  16-jim  square  photodiode  ele¬ 
ments  on  16-pm  centers  was  used  as  the  output  detector. 
This  circumvented  the  geometric  distortion  and  MTF 
problems  associated  with  vidicon  and  similar  detectors. 
The  array’s  27.6-mm  length  covered  ±114  cycles/mm 
of  spatial  frequency. 

To  simplify  the  output  electronic  system,  a  Tektronix 
3L5  spectrum  analyzer  with  571B  mainframe  was  used 
to  convert  the  video  signal  into  the  desired  correlation. 
In  this  system,  the  video  signal  v(t)  is  heterodyned  with 
an  oscillator,  and  the  product  is  bandpass  filtered  and 
peak  detected.  For  a  fixed  oscillator  frequency,  the 
final  output  is  the  amplitude  of  the  input  frequency 
component  at  the  difference  of  the  oscillator  frequency 
and  the  center  frequency  of  the  bandpass  filter.  As  the 
oscillator  frequency  is  swept,  the  output  is  the  spectral 
amplitude  density  of  the  input  in  time.  While  this 
system  does  not  explicitly  realize  Eq.  (5),  it  provides  an 
equivalent  output. 

Since  transform  lens  focal  length  and  detector  reso¬ 
lution  were  fixed,  the  input  size  had  to  be  adjusted  to 
accommodate  the  latter.  If  the  full  width  of  a  joint 
transform  input  is  36  (where  the  physical  size  of  each 
input  is  6  with  a  center-to-center  input  plane  separation 
26),  then  one  must  use  a  transform  plane  sampling  in¬ 
terval  of  u,  <  1/66.  For  various  practical  reasons  we 


used  us  =  l/12b,  oversampling  by  a  factor  of  2.  The 
detector  sampling  interval  of  16  Mm  results  in  us  -  0.132 
cycles/mm  and  a  nominal  input  width  of  36  =  3.77 
mm. 

To  demonstrate  the  use  of  this  system  in  the  corre¬ 
lation  of  pulse  coded  signals,  a  pseudorandom, 
twenty-pulse,  feedback  shift  register  code  was  generated 
by  a  computer  program.  The  bit  sequence  used  was 
1,0, 1,0, 1,1, 1,1, 0,0, 0,1, 0,0, 1,1, 0,1, 0,1.  Bar  pattern  input 
images  in  the  JTC  input  format  (see  left  side  of  Fig.  3) 
were  recorded  with  a  Xerox  Graphics  Printer  and 
photoreduced  200:1.  Actual  full  widths  ranged  from 
3.3  mm  (top)  to  4  mm  (bottom).  Input  patterns  with 
three  different  separations  corresponding  to  three  dif¬ 
ferent  signal  range  delays  and  the  corresponding  elec¬ 
tronic  analysis  system  outputs  are  shown  in  Fig.  3.  As 
predicted  by  Eq.  (5),  the  frequency  of  the  cosine  carrier 
increases  as  the  separation  between  the  inputs  (or  ef¬ 
fectively  the  target  range  delay)  increases.  The  corre¬ 
sponding  frequency  (horizontal  axis  in  Fig.  3)  or  hori¬ 
zontal  displacement  at  which  the  output  correlation 
peak  occurs  is  analogous  to  the  target  range,  time  of 
arrival  r,  or  shift  necessary  between  the  two  input 
functions  to  achieve  maximum  correlation 

p(r)  =  feU)f(l  +  r)dt.  (9) 

As  shown  in  the  right  side  of  Fig.  3,  the  location  of  the 
output  autocorrelation  peak  shifts  to  the  right  as  the 
separation  or  range  delay  between  the  input  signals 
increases.  The  middle  peak  represents  a  temporal 
carrier  frequency  of  150  kHz  in  u(t). 

Similar  experimental  demonstrations  of  this  corre¬ 
lator  were  obtained  for  text  and  aerial  imagery  inputs. 
The  results  for  the  aerial  input  case  are  shown  in  Fig. 
4.  The  objective  here  was  to  determine  the  location  of 
the  reference  object  g  (the  airfield)  in  various  input 
patterns  /.  This  pattern  recognition  operation  is  ap¬ 
propriate  for  a  missile  guidance  application.10  The 
shifted  location  of  g  in  /  in  the  sequence  of  three  ground 


Fig.  4.  Example  of  pattern  recognition  on  aerial  imagery:  (a)  input 
transparency;  (b)  correlation  output. 
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image  sensor  data  frames  shown  in  Fig.  4  is  intended  to 
represent  the  drift  of  the  projectile  or  missile  off  course. 
As  shown  in  Fig.  4,  the  coordinate  of  the  output  corre¬ 
lation  peak  is  proportional  to  the  location  of  the  refer¬ 
ence  object  in  the  input  data  and  hence  provides  the 
data  necessary  to  correct  the  flight  path  of  the  projectile 
or  missile. 

IV.  Equalizing  Correlator;  Theory 

In  the  conventional  JTC  system,  the  output  is  the 
correlation  of  the  two  input  functions  /  and  g  (with  no 
convenient  way  to  modify  the  operation  performed).  If 
these  functions  are  the  same  except  that  one  of  them, 
say  f(x),  has  been  acted  upon  by  a  linear  space-invariant 
system,  described  by  an  impulse  response  h(x),  then 
f(x)=g(x)*h(x),  and  the  correlation  output  from  the 
JTC  becomes 

/<*)©«(*)  =  [g(x>  *  h(*  )!©£(*>  =  h(x)  •  (/?(*)  ©g(x)|.  (10) 

The  effect  of  the  convolution  with  h(x)  shown  in  Eq. 
(10)  is  usually  a  degradation  of  the  output  correlation. 
The  correlation  peak  is  weaker  and  less  sharply  defined, 
and  the  output  SNR  is  less  than  that  of  the  autocorre¬ 
lation  case  g(x)  ©  g(x).  By  suitably  modifying  the 
basic  optical/electronic  correlator,  one  can  neutralize 
the  effect  of  this  type  of  signal  distortion  when  the 
distorting  transfer  function  is  known  a  priori  or  can  be 
estimated.  This  capability  follows  from  the  access,  for 
electronic  modification,  to  the  joint  transform  pattern 
video  signal  afforded  by  the  hybrid  correlator  system. 
Compensation  for  linear  time-invariant  distortion  is 
known  as  equalization  in  the  communications  field;11 
hence  we  refer  to  our  new  system  as  an  equalizing  cor¬ 
relator. 

To  see  how  this  equalization  can  be  accomplished,  we 
examine  the  video  signal  (ac  term  of  interest  only)  that 
results  when  g(x)  and  g(x)  *  h(x)  are  the  inputs: 

+  argf/^^y-^  j  +  etc.,  (11) 

where  H(u)  =  ,7[h(t)|  is  the  distorting  transfer  function. 
If  this  signal  is  fed  to  a  phase  modulator  while  a  tem¬ 
poral  voltage  waveform  — arg  H (sf  / X/t )  is  applied  to  its 
phase  control  input,  the  distorting  term  arg  H (st/Xfj,) 
is  removed  from  the  cos  factor.  Equivalently  we  can 
phase-modulate  the  heterodyne  oscillator;  this  latter 
method  is  easier  to  implement  electronically.  Similarly, 
if  v(t)  and  another  signal  |//(sf/A/j,)|_1  are  electroni¬ 
cally  multiplied,  the  product  will  be  free  of  the  multi¬ 
plicative  distorting  factor  |//|  in  Eq.  (11).  Fourier- 
transforming  the  resulting  signal  will  yield  the  auto¬ 
correlation  of  g. 

Arbitrary  linear  space-invariant  filtering  can  be 
performed  on  the  correlation  output  by  such  a  system; 
equalization  of  input  distortion  is  a  special  case  of  this. 
Since  the  system’s  filtering  function  is  determined  by 
a  pair  of  voltage  waveforms,  it  can  be  changed  with  the 
ease  and  speed  with  which  a  signal  generator  can  be 
reprogrammed.  The  equalizing  correlator  can  thus  be 


used  in  the  processing  of  radar  or  sonar  signals  degraded 
by  the  propagation  path  or  medium12  and  for  the  com¬ 
pensation  of  imperfections  in  optical  processor  system 
elements.  In  the  latter  category  are  nonuniform 
transform  plane  scanning  speed  and  a  restricted  class 
of  transform  plane  lens  errors,  namely,  those  equivalent 
to  applying  different  space-invariant  filters  to  the  two 
inputs  f  and  g.  If  the  distorting  function  is  known  a 
priori,  direct  correction  is  possible.  In  more  realistic 
situations,  some  knowledge  is  available  of  the  expected 
degradations,  or  various  degradations  can  be  assumed, 
and  the  system’s  performance  improvement  can  be  used 
to  judge  the  appropriateness  of  the  assumed  degrada¬ 
tion  on  an  a  posteriori  basis. 


V.  Equalizing  Correlator;  Experiment 

To  demonstrate  the  equalization  concept,  we  corre¬ 
lated  a  distorted  signal  against  the  original  signal  and 
applied  equalization  to  recover  the  correlation  peak. 
The  formatted  pair  of  inputs  is  shown  in  Fig.  5(a).  The 
original  signal  [left  side  of  Fig.  5(a)]  was  the  pseudo¬ 
random  pulse  train  used  in  the  autocorrelation  experi¬ 
ments.  The  distorting  transfer  function  H  was  of  unit 


Pig.  5.  Example  of  equalizing  correlat  ion:  (a)  phase-distorted  input 
and  reference  signals;  (b)  conventional  correlation  output;  (c)  equa¬ 
lizing  correlator  output. 
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magnitude,  while  its  phase  part,  shown  in  Fig.  6,  con¬ 
sisted  of  five  rectangular  pulses  of  magnitude  t  with  a 
baseline  of  zero.  The  distorted  input  pattern  was 
generated  by  numerically  convolving  the  undistorted 
input  with  h(x),  thresholding  the  resulting  signal,  and 
computer-graphically  printing  and  photoreducing  this 
hard-clipped  waveform  [right  side  of  Fig.  5(a)|.  The 
conventional  correlation  of  these  two  patterns  is  shown 
in  Fig.  5(b).  Positional  resolution  and  SNR  are  both 
seen  to  be  poor.  The  correlation  output  with  equali¬ 
zation  circuitry  in  operation  is  shown  in  Fig.  5(c) — a 
clear  correlation,  with  greatly  enhanced  SNR  and  po¬ 
sition-measurement  accuracy. 

The  particular  transfer  function  H  used  in  this 
demonstration  was  chosen  to  simplify  the  video  signal 
processing  circuitry.  The  distorted  video  output  signal 
was  fed  to  a  phase  modulator,  which  reversed  the  po¬ 
larity  of  the  video  signal  at  appropriate  times  to  realize 
modulation  by  — arg  H .  A  pulse  generator  was  used  to 
produce  the  five-pulse  sequence  (of  the  same  form  as 
Fig.  6)  that  was  used  to  control  the  modulator. 

This  equalizing  correlator  system  is  another  method 
by  which  the  flexibility  and  repertoire  of  operations 
achievable  on  an  optical  processor  can  be  increased. 
Matched  spatial  filter  weighting  and  synthesis  con¬ 
trol,2-13  hybrid  optical/digital  processing,14  and  space- 
variant  processing15-16  are  other  methods  by  which  the 
flexibility  of  an  optical  correlator  can  be  extended. 

VI.  Extensions  and  Discussion 

Another  use  of  the  general  optical/electronic  .JTC  is 
in  the  normalization  of  correlations.  Correlation  deg¬ 
radation  can  arise  when  one  of  the  input  signals  to  be 
correlated  is  reduced  in  strength  by  a  factor  a  from  that 
of  the  other  input  function.  The  degraded  joint 
transform  pattern  for  this  case  is  described  by  Eq.  (8). 
The  amplitude  2 a  of  the  cos  term  in  the  video  output 
signal  can  easily  be  measured  and  used  to  control  the 
vertical  scale  in  the  output  patterns  [Figs.  3(b),  4(b),  and 
5(c)).  This  results  in  a  normalized  output  correlation 
pattern  independent  of  changes  in  the  strength  of  the 
input  signal.  Other  nonlinear  processing  operations 
such  as  generating  the  logarithm  of  the  spectrum  (as 
required  in  cepstral  processing)  can  also  be  realized 
using  such  a  system. 

A.  Carrier  Period  Variance  and  Coherence  Measure 

The  presence  or  absence  of  a  correlation  and  its 
positional  coordinate  are  often  of  primary  concern 
rather  than  the  actual  shape  of  the  correlation  function. 
We  therefore  now  consider  another  new  optical/elec¬ 
tronic  system,  which  we  shall  call  a  coherence  measure 
correlator,  that  greatly  improves  the  speed  at  which  the 
correlation/no  correlation  decision  can  be  made  along 
with  ways  of  implementing  it.  We  conclude  this  section 
with  a  discussion  of  how  it  can  be  applied  in  range/ 
Doppler  multichannel  signal  processing.  This  system 
uses  the  configuration  of  optics  and  scanning  detector 
shown  in  Fig.  1.  However,  the  video  signal  is  now  op¬ 
erated  on  by  a  different  kind  of  electronic  processor 
than  that  discussed  so  far. 


■nwy 


Fig.  6.  Distorting  phase  transfer  function  used  in  the  equalizing 
correlator  experiment. 

As  was  pointed  out  in  Sec.  II,  phase  modulation  of  the 
fringe  pattern  causes  a  correlation  loss,  whereas  a  co¬ 
herent  uniform  fringe  frequency  (identical  inputs) 
produces  a  large  output  correlation.  Thus,  by  mea¬ 
suring  the  coherence,  or  spectral  purity,  of  the  video 
signal,  the  electronics  can  detect  an  autocorrelation,  or 
match  of  the  input  functions,  or  compute  the  degree  of 
similarity  between  them.  Several  qualitatively  related 
properties  of  the  video  signal  may  be  used  as  coherence 
measures,  namely,  carrier  period  variance,  carrier  phase 
modulation  or  phase  variance,  and  instantaneous  fre¬ 
quency  variance.  The  smaller  any  of  these  quantities 
are,  the  greater  the  video  signal  coherence  and  hence  the 
greater  the  degree  of  similarity. 

We  now  describe  how  the  carrier  phase  variance 
might  be  determined  by  electronic  postprocessing  of  the 
video  output  described  by  Eq.  (2).  Bandpass-filtering 
Eq.  (2)  removes  the  first  two  terms  and  leaves  2|F|  |G | 
cos[4irub  +  <t>(u  )|.  The  amplitude  modulation  present 
on  this  cosine  fringe  carrier  can  be  removed  by  a  limiter 
leaving  as  the  output  the  phase-modulated  carrier 
cos[47ru6  +  <t>(u)[.  If  this  signal  is  fed  to  a  phase  de¬ 
modulator,  we  obtain  a  voltage  output  proportional  to 
the  phase  modulation  <t>(u)  on  the  carrier.  Squaring 
this  quantity  and  integrating  it  electronically  over  a  line 
scan  time  yield  a  final  output  voltage  proportional  to  the 
carrier’s  phase  variance 

Xt  0 

<f>2(u)du  (12) 

ro 

If  a  frequency  demodulator  is  used  instead  of  a  phase 
demodulator,  we  obtain  the  instantaneous  frequency 
variance 

var|0'(u)|  =  C  |0'(u)|2rfu.  (131 

0 

To  compute  the  carrier  (fringe)  period  variance  over 
one  detector  line  scan,  we  threshold  the  video  signal  into 
a  digital  pulse  sequence  we  represent  by  (ui,  u  >, . . . ,  u,v) 
in  which  these  un  are  the  u  values  at  which  positive¬ 
going  zero  crossings  of  the  phase-modulated  carrier 
occur  [i.e.,  where  cos(4jt«5  +  </>)  =  0,  and  its  derivative 
with  respect  to  u  is  positive].  If  this  pulse  sequence  is 
then  fed  to  a  digital  or  combination  analog/digital 
processor,  the  mean  carrier  period 
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and  the  carrier  period  variance 
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can  be  calculated,  where  A u*  =  u*  —  u*_i  is  the  carrier 
period  at  the  fcth  cycle. 

This  electronic  processor  can  take  many  forms.  The 
mean  and  mean  square  pulse  width  over  one  line  scan 
time  can  be  calculated  simultaneously.  At  the  con¬ 
clusion  of  the  scan  we  subtract  the  square  of  the  mean 
from  the  mean  square  pulse  width.  This  yields  the 
variance.  An  alternate  simpler  circuit  to  measure  co¬ 
herence  is  the  phase-locked  loop  (PLL).17  A  PLL 
consists  of  a  voltage-controlled  oscillator  ( VCO)  and  a 
phase  comparator.  The  latter  compares  the  VCO’s 
output  with  the  input  signal  presented  to  the  loop,  and 
the  phase  comparator  output  is  connected  via  a  low  pass 
filter  to  the  control  input  of  the  VCD.  When  the  loop 
is  in  lock,  the  VCD  is  forced  to  track  the  input  signal, 
and  its  control  input  voltage  indicates  the  instantaneous 
frequency  of  the  input  signal.  The  FLL  thus  behaves 
as  an  FM  demodulator.  In  our  case,  the  ac  content 
(variance)  of  the  VCO  control  voltage  can  serve  as  a 
coherence  measure  when  the  video  signal  is  applied  to 
the  PLL’s  input. 

Coherence  measure  processing  requires  a  single  line 
scan  rather  than  repetitive  scans,  as  do  the  system  of 
Fig.  2  and  its  relative,  the  spectrum  analyzer.  The 
processor’s  output  is  computed  almost  immediately  at 
the  end  of  each  scan.  This  improvement  in  efficiency 
is  especially  important  when  the  2-D  nature  of  optical 
processing  is  exploited  in  performing  multichannel  1-D 
signal  processing. 

An  example  of  such  processing  is  the  correlation  of 
two  signals  differing  in  Doppler.  The  first  processing 
step  in  this  case  is  to  correlate  one  signal  against  a 
number  of  different  doppler  versions  of  the  other  signal, 
seeking  the  best  match.  The  second  step  is  to  extract 
the  range  from  the  correlation  of  the  optimally  similar 
pair  found  in  step  1.  This  processing  operation  can  be 
implemented  with  the  aid  of  the  coherence  measure 
correlator  as  follows:  On  successive  horizontal  lines  in 
the  input  plane,  signals  are  written  in  pairs,  with  one  of 
the  signals  repeated  and  the  other  signal  written  in 


different  doppler  versions  on  each  line.  This  array  of 
spatial  signals  is  Fourier  transformed  in  the  horizontal 
direction  and  imaged  in  the  vertical  direction  by  a 
spherical-cylindrical  lens  system.  The  horizontal  joint 
transform  fringe  patterns  on  successive  lines  are  sensed 
by  a  2-D  raster  scanning  detector.  The  coherence 
measure  parameter  is  computed  on-line  for  each  scan 
line  from  the  detector’s  video  signal.  The  line  which 
yields  the  greatest  coherence  indicates  the  optimal 
Doppler  match;  to  determine  range,  only  a  1-D  trans¬ 
form  (as  in  Fig.  2)  of  the  one  line  with  the  largest  co¬ 
herence  need  be  performed.  Thus,  range/Doppler 
processing  can  be  performed  in  one  scanner  frame  time 
by  a  relatively  simple  configuration  of  components. 

The  support  of  the  Air  Force  Office  of  Scientific  Re¬ 
search  on  contract  AFOSR  75-2851  administered  by  the 
Air  Systems  Command  and  the  Office  of  Naval  Re¬ 
search  on  contract  NR  350-011  for  the  work  reported  on 
herein  is  gratefully  acknowledged. 
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ABSTRACT 


A  classification  of  multi-sensor  imagery  from  the  sensor's  point  of  view  is  advanced.  From  this  treatment, 
the  statistical  and  deterministic  contributions  to  a  multi-sensor  image  correlation  process  are  more  clearly 
seen.  The  optimum  preprocessing  operation  for  several  cases  of  multi-sensor  image  pattern  recognition  are 
noted  and  the  use  of  weighted  matched  spatial  filter  synthesis  as  a  one  step  optical  pattern  recognition  cor¬ 
relator  is  described.  Theoretical  formulation  and  experimental  verification  of  the  result  that  edge  enhance¬ 
ment  preprocessing  is  not  always  optimum  in  a  multi-sensor  optical  image  pattern  recognition  system  are 
presented. 


1.  INTRODUCTION 

Performing  pattern  recognition  on  multi-sensor  (MS)  imagery  arises:  in  the  processing  of  satellite  imagery 
for  earth  resources  uses  [1),  in  the  registration  of  scenes  [  2,3],  and  in  missile  guidance  [  4,5]  among  other 
applications.  In  this  paper,  we  present  a  new  optical  pattern  recognition  (OPR)  approach  to  MS  image  pattern 
recognition  (IPR).  Our  primary  purpose  is  to  determine  the  agreement  between  a  given  stored  reference  scene 
f(x,y)  and  a  live  sensed  input  image  g(x,y).  For  the  particular  scenario  to  be  considered,  the  input  g  is  a 
radar  image  and  the  reference  f  is  an  aerial  image.  The  specific  problem  is  to  optimize  the  SNR  of  the  cor¬ 
relation  in  the  presence  of  typical  MS  image  differences. 

In  Section  2,  we  describe  a  new  model  for  MS  imagery  and  distinguish  three  cases  of  MS-IPR.  In  Section  3, 
we  discuss  the  optimum  preprocessing  necessary  for  each  case.  In  Sections  4  and  5  we  present  experimental 
confirmation  of  our  theoretical  formulation  and  discuss  our  resultant  data. 

This  new  model  of  the  MS-IPR  problem,  the  use  of  OPR  and  weighted  matched  spatial  filter  (MSF)  synthesis 
in  its  solution,  new  OPR  concepts  (such  as:  statistical  and  deterministic  OPR,  statistical  gradient  polari¬ 
zation  equalization),  and  new  issues  (such  as:  spatial  structure  distribution  and  gradient  quality)  represent 
the  major  new  contributions  contained  in  this  work. 

2.  STATISTICAL  AND  DETERMINISTIC  MS  IMAGE  MODELS 

MS  imagery  (with  all  geometrical  distortions  removed)  can  be  described  by  a  set  of  feature  vectors  that  are 
similar  in  both  scenes  (we  refer  to  these  as  key  objects)  and  by  a  set  of  different  features  (due  to  additive 
and  multiplicative  noise).  Most  PR  research  has  only  considered  additive  noise.  The  optimum  processor  for 
the  additive  white  noise  case  is  well  known  to  be  an  MSF.  In  MS  imagery,  additive  noise  arises  due  to  texture 
and  scene  detail  present  in  one  sensor's  image,  but  absent  in  another's.  Figure  1  shows  an  example  of  such 
an  image  pair. 

We  choose  to  include  multiplicative  noise  (characterized  by  image  contrast  reversals  of  random  polarity) 
in  our  description  of  MS  imagery.  More  so,  we  choose  to  describe  a  scene  from  the  sensor's  point  of  view  and 
to  clearly  distinguish  between  the  scene  itself  (no  sensor  variations  considered)  and  an  image  of  the  scene 
(obtained  with  a  given  sensor).  The  terms  scene  and  image  are  defined  as  above. 

If  we  consider  the  scene  itself,  it  can  be  modeled  as  uncorrelated  data  (if  it  contains  many  features).  In 
practical  PR  applications,  the  scene  contains  key  objects.  If  there  are  only  a  few  key  objects  and  if  they 
dominate  the  scene,  the  scene  is  highly  correlated  and  thus  more  deterministic  and  less  statistical.  As  the 
number  of  key  objects  in  the  scene  increases  (assuming  uncorrelated  and  independent  key  objects),  the  scene 
becomes  highly  uncorrelated. 

Contrast  reversals  are  the  dominant  unique  image  feature  germaine  to  MS  image  data.  We  thus  choose  to 
classify  MS  imagery  as  statistical  or  deterministic  solely  on  the  basis  of  contrast  reversals  and  to  model 
these  contrast  reversals  as  multiplicative  noise.  Thus,  we  will  adopt  definitions  of  MS  imagery  from  the 
sensor's  point  of  view,  rather  than  for  the  scene  itself.  If  a  pair  of  MS  images  are  dominated  by  contrast 
reversals,  we  refer  to  the  scene  as  statistical  (see  Figure  2).  We  note  that  the  distribution  of  the  polarity 
of  the  gradient  of  such  images  are  the  statistical  factor  to  which  we  refer.  This  leads  to  the  vital  aspect 
of  statistical  gradient  equalization  and  to  the  key  point  that  these  random  contrast  reversals  are  the  major 
generic  characteristic  of  MS  imagery.  In  some  instances,  neither  the  multiplicative  noise  (contrast  reversals 
in  Figure  2)  nor  the  additive  noise  (see  Figure  1)  dominate  and  the  resultant  MS  imagery  is  best  described  as 
deterministic  with  no  noise  (see  Figure  3).  The  MS  image  data  base  used  consisted  of  radar  and  aerial  images 
of  different  scenes.  The  three  image  pairs  to  he  reported  upon  are  shown  in  Figures  1-3  and  summarized  in 
Table  1. 
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Table  1.  Multi-Sensor 
Classification 
Deterministic  with  Noise 
Statistical 

Deterministic,  no  noise 


Image  Pattern  Recognition  Scenarios 

Pre-Processing 

Edge  enhancement 

band-pass , 
magnitude  detection 

Same  as  Auto¬ 
correlation 


Weighted  MSF 
* 

High  u 
Medium  u* 

Low  u* 


We  summarize  these  three  MS  image  cases  in  Table  1.  For  completeness,  we  note  the  optimum  digital  pre¬ 
processing  operation  required  for  each  case  and  the  corresponding  optical  weighted  MSF  synthesis  technique 
that  is  theoretically  optimum  for  each  case.  These  issues  are  discussed  in  Section  3.  In  these  MS  image 
definitions,  we  consider  only  scenes  containing  key  objects  (becasue  of  their  information  content),  but  we 
allow  contrast  reversals  to  occur  within  the  key  objects  as  well  as  within  the  scene’s  background. 

3.  THEORETICAL  BASIS  FOR  MS-IPR 


The  most  common  pre-processing  operation  performed  in  MS-IPR  is  the  edge  enhancement  or  differentiation  of 
the  input  images  [  2,3).  Such  an  isotropic  contour  operator  f  6]  is: 
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where  V  is  the  Laplacian.  This 
edges.  The  non-linear  isotropic 
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edges  of  the  scene  and  retains  the  polarity  of  the 
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retains  only  the  magnitude  of  the  edges  of  a  scene.  It  has  been  shown  [  20]  that  a  differentiation  pre-pro¬ 
cessing  operator  is  not  always  optimum.  Specifically,  for  highly  uncorrelated  scenes,  no  pre-processing  is 
optimum;  whereas  for  highly  correlated  imagery,  the  second  derivative  is  the  optimum  pre-processing  step  f 20  ] . 


When  noise  is  present  in  the  data,  the  above  conclusions  are  not  appropriate.  In  the  presence  of  additive 
noise,  the  optimum  pre-whitening  filter  can  be  shown  to  be  f  3 ] : 


h  = 


w 


—  u££(x,v) 

2A/xr  L 


{?  T*—  + 

<x 


(3) 


where  A,  a  and  B  are  parameters  describing  the  shape  of  the  noise  auto-correlation  function.  From  (3),  we 
clearly  see  that  a  derivative  or  gradient  pre-processing  operator  is  not  sufficient,  rather  a  weighted  combi¬ 
nation  of  no  pre-processing,  term  1  in  (3),  and  differentiation,  the  last  terms  in  (3),  is  seen  to  be  optimum 
when  noise  is  present. 


Svedlow  f  3]  found  that  a  threshold  gradient  pre-processing  operator,  in  which  the  polarity  of  the  gradient 
was  suppressed,  performed  best.  We  now  see  that  this  operation  is  preferable  because  it  removes  the  statis¬ 
tical  nature  (the  polarity  of  the  gradient)  of  a  MS  scene  (as  defined  in  Section  2).  Smith  [  7]  utilized  a 
linear  system  approach  to  the  MS-IPR  problem  for  the  case  of  additive  noise.  He  found  that  the  product  of  a 
noise  whitening  filter  and  a  weighted  filter  reflecting  the  target's  SNR  was  optimum.  These  combined  opera¬ 
tions  used  by  Smith  f  7|  can  be  shown  to  be  equivalent  to  a  type  of  bandpass  filter  rather  than  the  high-pass 
filter  (edge  enhancement)  normally  employed. 


Adjunct  spatial  filters  in  front  of  the  MSF  in  an  optical  correlator  can  perform  the  differentiation, 
bandpass  filtering,  and  noise  spectrum  whitening  operations  noted  above  ]  8,9]  .  In  optical  character  recog¬ 
nition  correlation  experiments.  Winze? r  f  8]  found  strong  erroneous  < ross-correlat ions  when  high-pass  filtering 
was  performed.  In  light  of  the  above  formulation,  we  can  sir  that  this  occurred  because,  in  his  optical 
character  recognition  case,  the  large  number  of  eharacters  in  the  input  search  scene  corresponded  to  our  case 
of  a  large  number  of  kev  objects  in  the  scene.  As  we  noted  in  Section  2,  the  scene  for  this  case  is  more 
statistical  and  less  deterministic.  For  such  cases,  d i if <rent i at i cm  pre-processing  alone  is  not  adequate. 

This  case  is  somewhat  analogous  to  the  missile  guidance  scenario  of  a  kev  object  surrounded  by  other  struc¬ 
tured  objects.  Synthesis  of  the  proper  phase  of  tin  differentiated  scones  (the  statistical  gradient  equali¬ 
zation  concept  noted  in  Section  2)  is  the  major  problem  in  such  adjunct  filtering,  schemes.  A  two  stage 
optical  system  (in  which  the  image  is  re-recorded  before  correlation)  could  be  used  to  perform  the  gradient 
equalization  step.  By  recording  the  magnitude  of  the  gradient  ol  tin  original  image,  the  polarity  information 
of  the  gradient,  and  hence  the  random  or  statistical  nature  ot  the  image,  is  removed.  We  refer  to  this  tech¬ 
nique  as  statistical  gradient  polarization  equalization.  We  will  concentrate  our  attention  on  a  single  step 
MSI-  optical  correlator  and  tin  use  of  weighted  MSF  svntlu-sis  to  optimize  the  NS-li’R  process. 
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The  concept  of  weighted  MSF  synthesis  is  quite  simple  l 10]  and  is  best  described  using  the  frequency  plane 
correlator  (11].  In  this  system  (Figure  4),  the  sensed  image  is  placed  at  Pj.  Its  Fourier  transform  G  is 
formed  by  Li  at  P2.  Stored  at  P2  is  the  MSF  with  nominal  transmittance  proportional  to  F*.  Leaving  P*  we 
find  F*G  and  at  P3  we  find  the  Fourier  transform  (denoted  by  the  operator of  this  product  of  two  Fourier 
transforms  or 


J  IF*G)  =  g  -*  f  ,  (4) 

which  is  the  desired  correlation  (denoted  by  the  symbol  )  of  the  two  functions.  Detailed  analysis  of  the 
MSF  synthesis  process  [10]  will  show  that  the  exact  transmittance  of  P::  is 


t  =  1  +  l/K  +  (2//K)  cos  (2ttocx  +  Arg  F)  ,  (5) 

where  a  is  the  spatial  frequency  determined  by  the  angle  between  the  signal  and  reference  beams  and 

K  =  A2/ | F | 2  (6) 

is  the  beam  balance  ratio.  In  (5),  we  note  that  the  fringe  modulation  is  a  maximum  when  K  =  1 . 


Figure  4.  Optical  Frequency  Plane  Correlator  Schematic. 

From  (6)  we  note  that  K  varies  spatially  (since  F  does)  and  hence  so  does  the  modulation.  Thus,  weighted 
MSF  synthesis  simply  involves  choosing  the  spatial  frequency  u*  we  wish  to  optimize  in  the  data.  Low  (high) 
u*  choices  result  in  equivalently  bandpass  (high-pass)  filtering  functions  for  the  MSF,  with  high  u*  values 
resulting  in  the  use  of  differentiated  images  in  the  correlation  process.  I  he  origin  of  the  entries  in  the 
last  two  columns  in  Table  1  should  now  be  apparent. 


A  final  aspect  of  MS-IPR  deserves  note  at  this  point.  We  refer  to  the  issues  of  edge  q ua 1 i t y  and  spatial 
structure  and  distribution.  The  first  term  refers  to  the  fact  that  the  derivatives  of  two  images  will  differ 
considerably  depending  upon  the  sharpness  of  the  edges  in  the  original  image.  As  a  result,  the  gradient  of 
one  MS  image  may  resemble  a  blurred  version  of  another  image  of  the  same  scene.  When  this  effect  occurs, 
statistical  gradient  polarity  equalization  effects  are  less  important  in  the  MS  image  correlation.  Hard 
limiting  is  one  often  used  pre-processing  step  for  such  MS  problems.  However,  this  operation  does  not  appear 
to  always  be  necessary.  This  conclusion  arises  because  the  low  and  mid-spatial  frequencies  associated  with 
the  relative  distribution  of  the  edges  of  the  scene  (i.e.,  their  relative  position  with  respect  to  one  another) 
are  preserved  in  all  cases.  We  refer  to  this  issue  as  spatial  structure  distribution  and  note  that  it  often 
contributes  more  to  the  optical  correlation  than  do  the  high  frequencies  due  to  the  edges  themselves.  This 
is  especially  true  in  practical  MS-IPR  data,  when  resolution  and  geometrical  differences  are  present  between 
the  two  MS  images  being  correlated. 

4 .  EXPERIMENTAL  CONFIRMATION 

Auto-correlations  and  cross-correlations  of  all  imagery  in  Figures  1-3  were  performed  with  K  set  equal  to 
1  in  four  different  spatial  frequencies  bands  (see  Table  2).  A  wedge-ring  detector  [12)  was  used  to  facili¬ 
tate  setting  of  K  values.  As  a  measure  of  correlation  plane  quality,  wo  use 


K2Mq.o)J_ 

;i  r  (  c  (  x ,  y  )  ! 


(71 


SPtt  Vo!  ?0 1  Optical  Pattern  Recognition  1 1979!  til 


-^MaimssssBa^ambMam 


CASASENT,  MUNOZ 


where  V?{  }  is  the  intensity  of  the  expected  value  of  the  correlation  c»  at  the  peak  at  registration  (0,0) 
and  where  correlation  noise  is  defined  as  the  variance  of  the  correlation  at  coordinates  (x,y)  far  from 
registration. 


Table  2.  MSF  Frequencies  Bands  Used 
Band  Spatial  Frequency  (cy/mm) 

B  2. 1-2.8 

C  3.4-4 . 4 

D  5.  1-6.4 

E  7.4-9. 3 

After  obtaining  power  spectra  data  for  all  imagery  (using  the  wedge-ring  detector),  auto-correlations  of 
all  images  of  each  scene  were  obtained  using  weighted  MSF's  with  K  ~  1  in  the  four  spatial  frequency  bands  u* 
indicated  in  Table  2.  For  each  scene,  both  positive  and  negative  radar  images  were  available  as  well  as  an 
aerial  image.  The  result  of  the  four  auto-correlations  (at  four  u*  bands)  for  the  image  in  Figure  2  are  shown 
in  Figure  5.  Auto-correlation  data  for  the  other  scenes  were  similar.  In  all  instances,  the  optimum  auto¬ 
correlation  was  found  to  occur  for  the  MSF  with  K  =  1  in  u*  band  B.  The  lower  SNR  of  the  radar  positive  image 
was  due  to  its  decreased  resolution  compared  to  the  corresponding  radar  negative  image.  From  Figure  5,  we 
also  see  that  the  auto-correlation  of  the  aerial  image  generally  yields  less  SNR  than  the  auto-correlations  of 
the  radar  images.  This  is  due  to  the  increased  mid-spatial  frequency  textural  data  present  in  the  radar 
images. 

These  auto-correlation  experiments  verified  our  earlier  remarks  on  the  more  deterministic  nature  of  an 
auto-correlation.  These  data  also  show  the  usefulness  of  weighted  MSF  synthesis  in  the  optimization  of  auto¬ 
correlations.  The  variations  of  the  correlation  with  u*  of  the  MSF  occur  due  to  the  statistics  of  the  scene 
itself.  Recall  that  neither  additive  noise  nor  phase  reversals  occur  in  the  auto-correlation  data  and  thus 
these  variations  in  SNR  with  u*  are  due  to  the  statistics  of  the  scene  itself,  rather  than  to  the  MS  features 
(such  as  auditive  or  multiplicative  noise). 


Figure  3.  Auto-Correlation  SNR  versus 
u*  for  the  Scene  in  Figure  2. 


Figure  6.  Cross-Correlation  SNR  versus 
u*  for  the  Three  MS  Scenes. 


The  resultant  optical  cross-correlation  experimental  SNR  versus  u*  data  are  shown  in  Figure  6.  These 
experimental  data  were  obtained  using  the  images  in  Figures  1-3.  In  all  cases,  an  MSF  of  the  aerial  scene 
served  ns  the  reference  with  the  radar  negative  as  the  input  pattern  (this  is  the  expected  MS-IPR  scenario). 
From  these  data  in  Figures  3  and  6  we  observe  that: 


(1)  Cross-correlation  SNR  can  he  optimized  by  proper  choice  of  u*. 

(2)  The  optimum  u*  for  the  cross-correlations  differs  from  the  u*  1 ound  lor  the  auto-corre lnt ions . 
( 1)  Different  u*  values  are  optimum  for  each  of  the  three  scene  classes  . 
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5.  DISCUSSION 


A  detailed  analysis  of  the  data  in  Figure  6  and  the  general  theoretical  remarks  advanced  earlier  in 
Section  3  and  summarized  in  Table  1  shows  excellent  agreement.  The  scene  of  Figure  1  (classified  as  deter¬ 
ministic  with  noise)  was  found  to  exhibit  the  optimum  cross-correlation  SNR  with  a  large  u*  setting  in  band  D. 
This  high  u*  value  is  equivalent  to  emphasizing  high  spatial  frequencies  in  the  image.  For  such  data  in 
which  additive  noise  is  the  dominant  MS  feature,  the  high-pass  filtering  preprocessing  step  should  be  mere 
heavily  weighted.  The  relative  weighting  is  determined  by  the  statistics  of  the  noise  as  in  (3). 

For  the  scene  in  Figure  3,  the  cross-correlation  SNR  was  found  to  peak  at  the  same  low  u*  hand  b 
setting  that  was  found  to  be  best  for  the  auto-correlation  case.  This  is  as  expected,  since  the  different  MS 
images  of  the  scene  in  Figure  3  do  not  exhibit  dominant  additive  or  multiplicative  noise.  We  thus  classified 
this  scene  as  deterministic  with  no  noise. 

For  the  scene  of  Figure  2  (classified  earlier  as  statistical  or  dominated  by  contrast  reversals),  a  mid¬ 
range  value  of  u *  equal  to  band  C  was  found  to  be  optimum.  This  mid-range  u*  setting  (larger  than  the  u* 
value  (band  B)  found  to  be  best  for  the  auto-correlation  and  less  than  the  highest  u*  setting  (band  D)  cor¬ 
responding  to  a  differentiation  or  high-pass  image  filtering  found  to  be  best  for  the  scene  in  Figure  1)  is 
representative  of  a  band-pass  filtering  MSF  or  equivalently  to  an  MSF  in  which  a  weighted  combination  of  no 
pre-processing  and  edge  enhancement  is  used. 

Another  important  aspect  of  MS-IPR  is  the  use  of  histogram  equalization  120),  In  some  MS-IPR  work,  Wong 
[17-18]  and  others  have  used  a  Karhunen  Loeve  amplitude  or  similar  image  pre-processing  operation  to  produce 
an  intensity  equalized  histogram.  Our  weighted  MSF  synthesis  technique  for  MS-IPR  can  be  reformulated  as  a 
type  of  frequency  domain  histogram  equilization  filtering.  By  decreasing  low  spatial  frequencies  and  in¬ 
creasing  high  spatial  frequencies  in  the  spectrum  of  the  image,  we  tend  to  equalize  the  histogram  ol  tie 
image’s  intensities.  Such  a  process  tends  to  flatten  and  broaden  the  spectrum  of  the  image  and  tends  to 
decrease  the  width  of  the  output  correlation  peak.  Such  issues  are  important  in  image  registration  ami  tor 
image  differencing  by  f rame-to-frame  correlation  [ 19] . 

We  used  SNR  defined  in  (7)  as  our  measure  of  correlation  quality.  By  whitening  the  spectrum  of  the  data, 
we  decreased  the  variance  of  the  correlation  noise  and  hence  increased  the  SNR  of  the  correlation  (decreasing 
the  low  frequency  components  in  the  image  decreases  the  statistical  correlation  contr ibut ions  and  this  makes 
the  correlation  more  deterministic).  To  determine  the  noise  filtering  properties  of  the  MSF  holograms,  wi¬ 
sh  ow  in  Figure  7  the  variance  of  the  output  correlation  noise  for  the  cross-correlat ion  of  the  imagery  of 
Figure  3  versus  the  optimum  MSF  band  u*  chosen.  As  shown,  correlation  noise  decreases  as  u*  increases,  thus 
indicating  that  flattening  or  broadening  of  the  spectrum  of  the  scene  occurs  in  weighted  optical  MSF  synthesis 


Figure  7.  Var(c(x,y))  versus  u*  for  the  cross-correlation  of  the  Figure  3  imagery. 

6 .  Sim\RY 

From  these  brief  theoretical  remarks  and  an  extensive  experimental  program,  we  have  shown  that  edge  en¬ 
hancement  pre-processing  is  not  always  optimum  for  MS-IPR.  Edge  enhancement  pre-process ine  is  optimum  onlv 
when  the  image  contains  a  well  structured  and  defined  key  object  and  when  the  MS  imaging  process  contributes 
additive  noise  to  one  MS  image.  The  true  MS-iPR  problem  corresponds  to  the  statistical  image  case  (Figure  2). 
In  this  instance,  the  contrast  reversals  in  the  imagery  contribute  to  tin  statistical  distribution  ot  tin 
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gradient  of  the  image  and  the  optimum  pre-processing  is  a  weighted  combination  of 
enhancement.  Such  multiplicative  noise  represents  the  true  MS  image  problem.  It 
the  standard  signal  and  additive  noise  problem  (Figure  1). 


no  pre-processing  and  edge 
differs  considerably  from 


Considerable  prior  MS-IPR  research  exists,  both  optical  [13-15]  and  digital  [3,16].  The  prior  IPR  research 
is  of  most  direct  concern.  In  [13],  radar  and  aerial  scenes  were  correlated.  In  the  correlation  process, 
emphasis  was  given  to  the  edges  of  the  image  (or  to  image  structure)  by  optical  differentiation.  However,  the 
major  concern  and  use  of  these  correlations  was  for  the  correction  of  geometrical  errors  in  the  image  pairs. 

In  [14],  a  general  formulation  of  the  optimum  edge  enhancement  frequency  plane  operator  was  advanced  and  (for 
certain  cases)  the  optimum  edge  enhancement  operator  was  found  to  be  similar  to  a  Laplacian  operator.  This 
analysis  lends  further  credence  to  prior  work.  In  [15],  major  attention  was  given  to  multi-spectral  data  and 
attention  was  focused  on  the  use  of  edge  enhancement  pre-processing  and  the  importance  of  the  number  of  key 
objects  in  the  scene.  In  this  formulation,  contrast  reversals  within  the  key  object  were  not  allowed  in  the 
definition.  In  some  digital  MS-IPR  research,  the  scenes  chosen  for  correlation  were  often  ll7,18]  dominated 
by  a  single  key  object  with  no  contrast  reversals  for  multiplicative  noise  present.  Such  images  correspond 
to  the  classical  signal  in  additive  noise  case,  not  to  the  true  MS-IPR  problem  involving  both  additive  and 
multiplicative  noise  (the  presence  of  contrast  reversals  within  the  scene  as  well  as  within  the  key  object). 


In  all  cases,  no  prior  work  of  which  we  are  aware,  has  addressed:  the  statistical  and  deterministic  contri¬ 
butions  to  the  optical  MSF  pattern  recognition  correlation  process,  the  importance  of  the  polarity  of  the 
gradient  of  the  scene  in  the  OPR  of  such  MS  data,  and  the  issues  of  image  edge  quality  and  image  spatial 
structure  distribution.  In  our  work  reported  upon  here,  we  have  shown  both  in  theory  and  by  experiment  that 
an  optical  correlator  using  weighted  MSFs  can  successfully  correlate  MS  imagery.  We  have  also  shown  that  such 
a  pattern  recognition  system  is  of  use  in  cases  of  multiplicative  noise,  additive  noise,  and  no  noise.  This 
research  was  conducted  for  one  specific  MS-IPR  data  set  (aerial  and  radar  images).  However,  the  results  appear 
to  be  general  enough  to  be  of  use  in  other  MS-IPR  cases  or  other  IPR  applications.  A  new  model  for  such 
imagery  and  new  definitions  of  classes  of  such  imagery  as  well  as  the  optical  pre-processing  required  for  each 
case  have  also  been  defined  in  the  course  of  this  work. 
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A  hyporspaco  description  o(  the  inidlii  loss  (multiobjocl)  poltern  recognition  problem  is  advanced.  Average  filters 
composed  of  linear  sums  of  ortlionorma!  basis  functions  are  found  to  provide  the  necessary  discriminant  hypersurfaces. 
Optical  correlation  using  weighted  matched  spatial  filter  synthesis  is  used  to  determine  the  basis  functions  and  their  linear 
weights.  Tlie  average  filters  are  then  assembled  on  a  digital  computer  and  used  in  an  optical  frequency  plane  correlator. 
Initial  problem  formulation  theory  and  simulation  results  are  included  for  an  infrared  tank  pattern  recognition  problem. 

Introduction 

In  the  general  pattern  recognition  problem  multiple  classes  of  objects,  whether  inherent  or  generated  by  a  specific 
object,  must  be  recognized  with  rejection  of  other  objects  not  in  the  multiclass  set.  The  rnultiaspects  of  targets  encountered 

1-3  4 

during  target  acquisition  for  missile  guidance  as  well  as  product  line  inspection  are  diverse  examples  of  this  general 
multiclass  pattern  recognition  problem.  In  this  paper,  a  description  of  hyperspace  (next  section)  and  its  use  in  solving  this 

pattern  recognition  problem  is  reviewed5.  Ve  show  how  the  discriminate  hypersurfaces  necessary  to  accomplish  multiclass 
recognition  and  provide  adequate  false  target  rejection  can  be  achieved  by  using  an  average  filter.  In  the  section  on  basis 
function  determination,  we  describe  how  these  average  filters  can  be  produced  as  weighted  linear  combinations  of  the 
orthonormal  functions  (the  hyperspace  basis)  by  which  the  input  and  reference  have  been  described.  How  optical  correlation 
using  weighted  matched  spatial  filter  synthesis  cun  be  used  to  determine  the  basis,  their  weights  and  the  average  filter  is 
discussed  in  the  section  entitled  lit  tank  pattern  recognition  together  with  experimental  data  and  a  complete  correlation 
matrix. 

The  experimental  case  chosen  is  the  recognition  of  a  tank  from  lit  imagery  independent  of  the  aspect  of  the  tank.  Our 
initial  average  filter  results  and  the  result  of  correlations  between  it  and  various  aspects  of  the  tank  target  are  presented  in 
the  section  on  Ili  tank  pattern  recognition. 

Prior  approaches  to  the  multiclass  problem  have  involved  the  use  ol  multiple  matched  spatial  filters5’**.  Little 
attention  has  been  given  to  pattern  recognition  techniques  lor  Ili  sensors  although  considerable  Ili  sensor  advancements  have 
7-9 

occurred  .  Other  researchers  have  suggested  post  processing  of  the  correlation  matrix  outputs  of  a  multichannel  correlator 

to  enhance  discrimination  in  cases  such  as  character  recognition*15.  Although  our  research  is  of  use  in  this  latter  area,  our 
present  concern  is  to  improve  the  recognition  of  a  multirlass  set  of  objects  moreso  than  discrimination  of  one  particular  class 
from  another  class.  The  average  filter  used  is  based  on  the  correlation  matrix,  thus  incorporating  post  processing  into  the 
filter  itself,  and  avoids  the  need  for  multiple  filters  and  extensive  jxrst  processing. 

Hyperspace 

We  will  consider  one-dimensional  functions  for  simplicity  only.  We  denote  the  input  image  as  f  and  assume  that  K 
different  inputs  can  occur.  The  object  functions  to  be  recognized  arc  g  of  which  we  assume  that  N  different 

orientations  jsn|  it  are  available.  The  average  filter  h  to  he  constructed  will  be  a  weighted  linear  combination  of  the  gn- 
The  task  of  the  pattern  recognition  system  is  to  recognize  I  li  i  belongs  to  the  set  of  functions  j  g^  j  and  to  reject  it 

otherwise.  The  K  inputs  may  he  different  orientations  of  I,  but  not  necessarily  one  of  the  N  orientations  in  lg|(t  .  In  fact,  in 
the  i:c  ‘nor.  1 1  for  mol.  1 1  ion,  I  he  I  e  I  need  no  I  lx*  <h  Morel  it  of  ii-nl.if  iom,  ol  one  obior  I  g.  ? 

("I  . 

We  begin  the  dev  ripllon  ol  our  svslom  by  expanding  I  in  a  sot  ol  ortlionoi  null  dilu  tions  s/>,  i.o., 

I  (x)  *  a  l|'(  (x)  (I) 
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in  a  multidimensional  vec  tor  space  (hypcrspuoe)  where  the  0(x)  are  the  basis  lain  lions.  One 
expanded  similarly  * 
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The  correlation  output  is 
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(0 


(5) 


A  A 

From  (5),  we  have  established  the  special  significance  of  the  dot  product  as  the  correlation  of  f  with  g  in  this  representation. 


To  illustrate  the  fundamental  ideas  of  discriminate  surface  generation  in  hyperspace,  we  consider  the  two-dimensional 
diagram  of  Figure  I.  fiach  X  denotes  a  vector  belonging  to  the  set  of  objects  to  be  recognized  and  each  O  is  an  unwanted 
object  to  be  rejected.  In  this  simple  case,  many  discrimination  surfaces  are  possible  because  the  X  and  O  points  are  well 

A  A 

separated  and  clustered.  Since  f.g  =  constant,  where  g  is  a  fixed  vector,  describes  a  line  (hyperplane)  in  the  diagram,  a  single 

✓\ 

average  filter  g  can  be  used  followed  by  threshold  detection.  Alternatively,  surface  I  could  be  chosen  and  generated  by  f*f  s 
constant,  a  hypersphere.  In  this  case,  surface  1  is  an  equi-energy  surface  and  a  simple  criteria  such  as  requiring  the  auto¬ 
correlation  peak  of  the  input  to  be  within  a  fixed  range  will  provide  adequate  discrimination.  Surface  III  is  another  more 
complex  possibility.  However,  it  is  more  difficult  to  produce  requiring  many  average  filter  surfaces  and  a  multichannel 
system  to  approximate  it.  Such  techniques  are  necessary  as  performam  e  requirements  are  increased  and  are  essential  when 
the  input  data  has  low  SNR. 


Our  interest  is  in  the  multiobject  case  which  we  illustrate  by  considering  the  tw<o  vectors  fj  and  f^  of  Figure  1.  The 
pattern  recognition  problem  is  to  determine  if  cither  f(  or  I?  is  present  at  the  input  and  reject  all  other  objects.  Both  f(  = 
(a j  j,  a^)  and  I2  =  j >  a 22 )  lle  on  a  line  (plane)  which  can  be  specified  by  f  ■  g  r  constant.  In  particular,  g  can  be  seen  to  be 
the  vector  normal  to  the  plane  and  the  constant  is  -jg.gl  Thus  g  has  been  uniquely  specified  and  is  found  to  be  a  linear 
combination  of  ((  and  f^,  i-e., 

0.  ~  /s  ■'s  0 

g  =  bj  fj  +  bj  f2  =  <b|  3j  |  +  b|  a^)  0|  *■  (b2  a2j  *  ^2  a22^  ^2  (6) 


Thus  with  g  and  a  simple  thresholded  output  the  task  is  accomplished.  This  can  be  extended  to  more  dimensions  as  outlined 
below  arid  an  average  filter  h  defined  to  perform  multiobject  recognition. 


Basis  function  determination 


We  now  consider  how  the  basis  functions  0  and  the  average  filter  h  can  be  found.  For  this  case,  we  consider  N 


reference  functions 


I6") 


given  by 


gn(x)  =  “  bnj  <*.(x> 


The  | gn  j.  will  be  recognized,  if  they  are  in  the  set  |  fn  j,  of  inputs,  by  an  average  filter 
h(x)  =  I  o  (x). 

The  correlation  outputs  for  the  multiclass  pattern  recognition  case  are 

R  ,  (o)  :  R  (o)  =  8  #h  :  "  b  .  c  ;  l( 
g  h  n  hn  1  m  1  n 

"n 

where  (2),  (7),  and  (8)  have  been  user).  The  objective  is  to  find  (  ,  b  and  finally  the  e  's  ol  h  such  that  R 
the  acceptable  correlation  performance.  I  I  I  111  1  gnh 


(7) 


(S) 


(9) 


provides 


We  have  shift  invariance  and  can  assume  that  the  n  <  urrelations,  R  ,  peak  at  /eie  lor  all  legistered  inputs.  This  "co- 
location"  feature  will  require  shifting  each  g(  to  the  <  on  e<  t  loi  at  ion  when  lot  uuug  h  and  is  am  opt  able  sun  e  this  is  an  ult¬ 
imo  proi  edure.  This  feature  allows  the  weight  mg  ol  |X'it  ions  ol  eat  h  g  ill  I  lei  cut  l\  w  lieu  lm  uuug  h. 


7,V  /  SI’/I  Vul  .1)1  Ollln  .ll  I'.lllrin  infill 


/ 


t  Jl1 1  li, A!  I'AI  I  min  HI  CuCNMION  IJSINii  AVI  IIACiL  III  II  Hl>  III  I'M  I  iim.IUMINAN  I  Hrl'l  HbUlil  ACI  S 


If  the  Gram- Schmidt  *  1  procedure  is  used  to  select 
pairs  g.  *9  g^  of  jg^j,.  The  relationships  are: 

(J>l  (x)  g(  (x)/k j 

(x)  [  g,  (x)  -  R  |  j  O |  (x)]  /k? 


then  tee  find  the  's  froiti  the  correlation  matrix  R  .  of  the 
)  i) 


n  -  I 


(x)  [  g  (x)-  i  R  t/t  (x)  l/k 

n  1  ''n  .  nil  1  n 


(10) 


where  the  k  are  normalizing  constants  that  are  functions  of  the  R  .  With 

"  °  i) 


jv/i^j  dote 


determined,  the  coefficients  b  .  in  (7)  are 

ni 


found.  We  can  then  solve  the  set  of  N  cross-correlation  equations  in  (0)  tor  c  .  For  the  mult ir  lass  case  the  equations  for  R 

1  i  i  " 

in  (9)  are  all  set  equal  to  a  constant.  Specific  examples  of  the  determination  ot  the  I'  correlation  matrix,  of  ( if'  s  ,  of  b 
and  It  follow  in  the  next  two  sections.  ')  I  I  f  ni 


IR  tank  pattern  recognition  (determination  ol  correlation  matrix  R  ) 


The  particular  pattern  recognition  cast1  study  *  hosen  to  utilize  the  te<  Imiques  advanceii  in  the  second  anti  third  sections 
of  this  report  was  the  acquisition  of  a  military  lank  target  irom  IR  sensor  ui.ageiv  independent  of  the  target's  orientation. 
The  imagery  was  taken  through  the  S-14  micrometer  IR  window  because  ot  the  higher  and  more  reliable  radiance  image 
variances  of  this  window.  The  seven  tank  aspects  used  as  the  I  I  reterem  e  set  are  shown  in  Figure  1 

|  "I 


To  generate  the  correlation  matrix  R  .  the  seven  lank  images  were  correlated  using  a  matched  spatial  filter  (MSF) 

12  13 

correlator.  The  MSI-  technique  '  performs  a  bandpass  Idler  in  the  correlation  pro<  ess.  [3y  vary  mg  the  spatial  Irequency  u* 
at  which  the  beam  balance  ratio  K  (ratio  ol  the  reterem  e  to  signal  beam)  is  unity,  the  li.indp.iss  an  be  varied.  The  resultant 
peak  intensity,  1^,  of  the  optical  I  orrelation  ol  e.e  h  ol  thi  1 1 V  •  •  mo|of  tank  ve-ws  plotted  against  u'  is  shown  in  Figure  3. 

From  these  data,  the  optimum  bandpass  tiller  was  Inuim  to  lie  >■.,  with  o'  ?./’>  ■  y/mm.  This  u'  was  <  Imsen  lot  all  future 
MSF's. 


All  pairs  of  the  seven  images  jgr|J  111  I  'gute  2  were  then  ante-  and  c  ross-correlated  both  optically  (using  the  MSI7 

techniques  used  to  determine  o')  and  digitally  (using  mu  ii.m-al  t  .vo  ui  mens  ion.,  I  I  II  tc<  l.niqurx  on  the  same  images  which 

were  digitized  and  bandpass  filtered  at  the  digital  Irequen.  v  u  as  g>..v-).  In . .  u  e  v :  ess-correlat  :.m  'natrix  is  show  n  in 

Figure  4. 


The  correlation  coefficients  R  Iron  Figure  at. 

i) 

Schmidt  procedure  of  (10).  Shown  m  I  igm e  f.iir  the  tea.! 
-  .(.'*2  x  g, 

and  inversely  g|  |.5gt/>j.  Furthermore,  with  Hie  del. u  i, 
be  the  inverse  ot  the  table  in  li^ure 


■»'•••  1  t'1  i  ate  Hu-  basis  turn  tions  b\  the  Grum- 

' t  'f  <  •  \»i.  •  j  •!»  ■.  t  f . ’*i :  I  i  j;  ir  r  '* 

(II) 

ll  ’  «  iV'*-  l< ''  vvi'9'''ls  b  lf  in  (7)  .in-  determined  and  found  to 


II*  tank  pattern  in  oi’jnt  i ■  i-  fdeter  ituiMl  ion  ol  h) 

The  coefficients  c  of  tlx*  average  filter,  li,  in  (X)  were  inn  ■  .imputed  with  the  appropriate  shifts  in  the  as  noted  tn 
the  third  section.  The  resultant  average  lilter  relative  w i  i>;iii  mf;  .  . 


h  r  0.64  */> 


0.60  t/>2  ♦  0.42  (/>  ^  .  0.93  .  Q.V/  .  O.'i 


I.X‘> 


(12) 


A  reconstruction  ol  the  average  lilter  obtained  is  shown  in  l  i.nif  The.  1 1 1 r *  i  w.r.  then  .or  related  with  all  seven  ol  the 
input  images  in  Injure  2.  The  |M%al\  intensities  ol  all  the  «  or  i  rl.d  h'if.  were  in  i  ranee  Iron.  seven  to  30.  \  projet  t  ion  ol  five 

*  ross-sei  tiohal  scans  around  the  peak  ol  a  typical  output  i  owek,ti.»n  .  o!  g  h  is  shown  in  f  igure  7.  MI  seven 

correlation  surfaces  exhibited  a  sharp  »  orrelation  peak. 


Xuunn.n  \ 

In  pattern  recognition  problems  where  a  mult  lass  ot  objt  .  ts  must  he  ici  .\i;ii;,vi|  as  one  <  lass  of  objects,  we  have 
shown,  rising  a  hyporspace  formulation,  that  t h» ■  umlti<  lass  «Iim  t  immnte  surl.ne  <  an  he  generated  usmi’  an  average  lifter. 
l:rom  our  hy|>etNp.ne  repf  esenlnlton,  a  luelliod  ol  *  al<  ulat  h»n  ol  .a-  li  an  avei  ape  lill-u  was  duet  tl\  developed  based  on  an 
otliurnormal  I  unction  expansion.  I  ur  lher  more,  the  'Method  was  >l>-mi*nstr  ale<!  unm*  optica!  pio<esstnp,  technKjues  on  an  II’ 
(Ml  tern  recognition  problem  assoi  luted  With  taipel  a<  ijuisil  i  or »  id  i  tank  al  moll  i.ispei  ts.  \  VUam-St  hnudt  expansion  Was 
ust-d  as  the  basis  (millions  and  the  resultant  aveiatv  lilter  was  .  orndited  w.th  .,n  im.ijv  set.  I  he  Idler  exhibited  pood 
per  b  >1  u  lain  e  with  the  •  oi  i  ela  I  ion  si  it  I .  m  r>  t  lieur.r  |\  r,  j  tosses  .r  r  the  ii  ■  >|  h  *t  t .  in  t  «  bar  n  I  <  li  .  1 1>  si  i.a  p  peak  id  <  oi  i  olat  ion 
(lidet  |  ion. 
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FIGURE  1.  TWO-DIMENSIONAL  SPACF 
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Abstract 


A  novel  approach  to  coherent  optical  pattern  recognition  is  described.  It  does  not  utilize  matched  spatial 
filters  and  optical  correlation  methods.  Rather,  a  2-dimensional  input  function  is  described  in  terms  of  its 
absolute  normalized  invariant  moments.  Comparison  of  these  moments  enables  one  to  determine  the  presence  of  a 
given  object  independent  of  geometrical  distortions.  A  parallel  optical  processor  with  a  special  mask  is  used 
to  generate  all  moments  for  a  2-D  scene  in  parallel.  A  simple  digital  post  processor  calculates  the  actual 
absolute  normalized  invariant  moments  with  the  high  dynamic  range  necessary.  Our  initial  work  reported  upon 
here  has  concentrated  on:  the  optical  generation  of  individual  moments,  how  the  individual  moments  vary,  the 
dynamic  range  requirements  of  the  system  and  how  they  may  best  be  met  in  a  hybrid  optical  digital  topology, 
and  methods  to  generate  all  moments  optically  in  parallel. 

1.  Introduction 


The  flexibility  of  coherent  optical  pattern  recognition  systems  and  the  repertoire  of  operations  possible 
on  these  systems  have  greatly  improved  in  recent  years  II].  Optical  pattern  recognition  has  long  relied  on 
the  Fourier  transform  and  matched  spatial  filter  correlation  operations  inherently  possible  in  an  optical 
computer  [2].  The  purpose  of  the  present  research  program  was  to  develop  a  new  approach  to  optical  pattern 
recognition  that  did  not  rely  on  the  Fourier  transform  and  matched  spatial  filter  systems  that  have  been  used 
for  so  long.  Recent  research  in  matched  spatial  filter  [3]  and  space  variant  [4]  optical  pattern  recognition 
has  concentrated  on  practical  pattern  recognition  problems,  namely  maintaining  correlation  when  the  input  and 
reference  are  not  equal.  Geometrical  differences  between  the  input  and  reference  scene  are  the  major  cause  of 
correlation  performance  degradation. 

In  this  paper,  we  describe  a  novel  coherent  optical  pattern  recognition  technique  that  appears  to  allow 
pattern  recognition  independent  of  any  geometrical  differences  between  the  input  and  reference  scene.  The 
technique  we  employ  is  to  represent  an  image  by  its  absolute  normalized  invariant  moment.  This  approach  has 
its  origin  in  the  mathematical  literature  on  algebraic  invariants  [5]  and  moment  invariants  [6].  Several 
recent  digital  processing  programs  in  this  area  have  been  reported  [7-9].  These  recent  works  have  shown  the 
usefulness  of  this  technique  in  pattern  recognition,  but  the  computational  load  involved  in  the  calculation 
of  the  moments  has  impeded  further  progress. 

In  this  paper  we  present:  a  less  mathematical,  revised  formulation  of  the  theory  of  invariant  moments 
(Section  2);  a  detailed  numerical  case  study  using  a  simple  square  input  function  (Section  3);  the  design  and 
fabrication  of  a  coherent  optical  system  to  generate  individual  moments  for  a  2-1)  it. put  pattern  and;  initial 
experimental  results  obtained  on  our  system  (Section  4).  An  optical  system  to  generate  all  moments  in 
parallel  is  then  briefly  discussed  together  with  an  analysis  of  the  required  system  dynamic  range.  This  re¬ 
sults  in  the  design  of  an  optimum  hybrid  optical /digital  system  for  pattern  recognition  using  absolute  normal¬ 
ized  invariant  moments  (Section  5). 


2  .  Thetiret  U  a  1  bas  i  s 

The  theory  of  algebraic  invariants  (3)  begins  by  the  description  of  a  homogeneous  polynomial  of  u  and  v  as 
a  binary  algebraic  form  or  more  simply  ns  a  binary  form  of  order  p 


,.p-i 


The  more  compact  notation: 


(>o;  ap-i,r  •••  :  ai,P-i:nory  p 


)  H' 


(1) 


(2) 


for  this  polynomial  is  preferable.  This  homogeneous  polynomial  of  the  coefficients  a  ...  is  said  to  be  an 
algebraic  invariant  of  weight  w  if: 


1  (a 1 . a  *  )  =  \w  I  (a  „ . a.  )  , 

pO  Op  p0  Op 

where  a^  are  the  coefficients  of  the  polynomial  expansion  after  the  general  linear  t ransf ormat ion 


t  > 


*  0. 


(3) 


(4) 


SPIE  Vof  20 1  Optical  Pattern  Recognition  (7979/  107 


f 


CASASENT.  PSALTIS 


In  (4)  A  is  the  determinant  of  the  coordinate  transform  and  for  absolute  invariance  we  require  w  =  0.  The 
basic  idea  associated  with  algebraic  invariants  is  to  describe  a  general  geometrical  distortion  in  terms  of 

(4)  and  to  then  determine  the  relationship  that  the  coefficients  a  and  a'  must  satisfy. 

pn  pn  J 


The  moment  generating  function  of  a  2-D  input  function  f(x,y)  is 

CO 

M(u,v)  =  exp(ux+vy)  f(x,y)dxdy 


(5a) 


l  l 

p=0  q=0 


pq 


q: 


(5b) 


■// 


\  (ux+vy)1*  f(x,y)dxdy 

p=0 


(5c) 


p=Q 


h  (mPo . m0p>(u’v>p 


(5d) 


This  operator  is  most  useful  in  determining  the  moment  invariants.  The  integral  form  in  (5a)  is  the  funda¬ 


mental  definition.  The  moments 


pq 


of  a  2-D  function  f(x,y)  are  defined  by 


pq 


-  J f  x^y^f (x,y)dxdy  ,  p,q  =  0,1,2. 


(6) 


From  this  definition  we  can  express  M  as  a  power  series  expansion  of  the  moments  m  as  in  (5b).  Expanding 
the  exponential  term  in  (5a)  in  a  power  series,  (5c)  results.  Interchanging  the  o£§er  of  integration  and 
summation,  the  compact  notation  [6]  in  (5d)  results. 


The  general  formulation  procedure  to  determine  the  invariant  moments  uses  the  algebraic  invariants  and 
moment  generating  function  concepts.  The  basic  .approach  is  to  apply  the  general  linear  transformation 


L. 


M 

i 

I  yi 

L 


(7) 


L  _l 

to  f(x,y).  A  new  Mj  moment  generating  function  results.  Applying  the  coordinate  transformation  in  (4)  to  M, 
we  obtain  a  new  Mq(u',v')  of  the  same  functional  form  as  M.  Rewriting  M  and  M  in  the  form  of  (4)  and 
equating  coefficients,  the  relationships  necessary  to  make  Mpq  and  M'  (the  moments  of  the  original  and  co¬ 
ordinate  transformed  functions)  equal  are  obtained.  The  proper  combinations  of  several  moments  can  then  be 
selected  such  that  they  are  independent  of  the  indicated  coordinate  transformation.  This  general  procedure 
thus  involves  application  of  (7)  to  f(x,y)  to  yield  f,(x',y')  and  a  new  M}  and  application  of  (4)  and  (7)  to 
(5c).  This  yields  a  moment  generating  function 


M^u'.v’)  = 


'// 


l 

p=0 


(u’x’+v’y*)  f(x',y')(l/|j| )dx'dv ' . 


(8) 


where  J  is  the  Jacobian  of  the  coordinate  transformation  in  (7).  (For  a  linear  coordinate  transformation, 
J=A,  the  determinant).  Note  that  application  of  (4)  and  (7)  to  (5c)  is  equivalent  to  the  application  of  (7) 
to  (5c)  with  the  coefficients  of  x'  and  >■'  in  the  coordinate  transformation  factor  (ux+vy)  equal  to  u'  and  v' 
Note  also  that  the  choice  of  the  coordinate  transformations  in  (4)  and  (7)  are  such  that 


(ux+vy)  =  (u'x'+v'y'). 


(9) 


Since  M  and  M|  are  of  the  same  form  as  in  (5c)  and  (8),  wc  can  express  them  as  in  ( 5d )  .  1,'sing  (9),  t  ho  co¬ 
efficients  in  these  two  coordinate  transformed  versions  of  M  and  Mj  can  he  compared  and  the  moments  of  order 
p  have  the  same  invariance  as  in  (3)  with  an  added  factor  J  as  in: 


i(mP<r 


V  =  |JlAW’(mPo . "V- 


(10) 


The  basic  conclusion  Is  the  formulation  in  (10)  and  the  general  procedure  described.  Iso  of  this  technique 
is  best  demonstrated  by  example.  This  is  provided  in  Section  3. 


3 .  Case  s t ud v 


Specific  cases  of  translation,  scale,  and  rotation  invariance  are  analyzed  in  general  terms  in  Sections 
3.1-3. 3.  The  absolute  norma li zed  invariant  moments  (Section  1.4)  then  result. 
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3*1  Translational  invariance 


Under  the  coordinate  transformation  (11) 


x  ’  -  x-fu 


y’  =  y+3 


a, 3  =  constants. 


where  a  and  3  are  the  shift  variables,  the  central  moments  u 


(x-x)1 (y-y)4f (x,y)d(x-x)d (y-y) , 


where  the  centroid  of  f  is  described  by 


x  =  mio/nW 


y  mQ1 /m0Q 


are  invariant  under  a  coordinate  translation.  The  central  moments  u  can  thus  be  expressed  in  terms  of  the 
ordinary  moments  m  (for  the  first  four  terms)  by 

pq 


‘‘00  ~  00* 


,J10  "  °'  "30  =  m30‘3xra?0+2n,10x 

>J01  =  °’  "12  =  m1  2-2yml  1  ■xm02+2>’2ml  0 

"20  '  m20_Xm10  |:21  =  ni21-2xmu->’™20+2x  m01 

"02  =  ra02_>™0l  1,03  =  'n03'3>7m02+2y“m01 

It  can  be  shown  (6|  that  no  differences  result  if  M  is  described  using  }J  rather  than  m. 
3 . 2  Sea 1 e  in va r iance 

Under  a  scale  transformation  (scale  factor  a) 


=  j  ,  a  =  constant ,  ( 1  >) 

v  ’  n  .1  v 

l  -  L  _  ~  J 

the  moment  generating  functions  M  and  can  i>e  expressed  in  the  form  of  (5b).  Equating  eoeft  iciint:.  in  these 
expressions,  we  find  the  moments 

m 

•'  =  JK'-  ■  (16) 

pq  i+(i>+d)/2 
moo 

to  be  invariant  to  a  scale  change  and  the  moments 

U 

=  ---PA  --  07) 

•pq  l+(p+q>/2  u/; 

■'no 

to  be  invariant  to  both  a  scale  and  translational  change, 
i. 3  Ko tat  Iona  1  invariance 


The  rotaLional  transformation 


in  "I  r*l 


Lv  'j  f  -s  i  n  1  cos  ■  j  v  j 

'  J  i-  J  I _ i 

c;in  be  achieved  by  describing  M  and  Mj  in  terms  of  i.  and  m  and  .  1  *  and  m1  in  the  form  of  (3b).  Comparing  co¬ 
efficients  between  m  and  m*  or  ,1  and  in  these  M  ami  Mj  equations  yields  the  desired  moments.  Re  I  ormu  lat  ion 
of  the  coordinate  transformation  hv  proper  redefinition  of  (u,v)  axes  greatlv  simplities  the  procedure.  The 
resultant  roLvition.il  invariant  moments  become 
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P 

P 

P 

P 

P 


1 

2 

3 

4 

5 


=  m2()  +  m02 
=  (m2o-m02)2  +  4mu2 

^m30-3m123  +  ^3m2l“m03^ 

=  (m30+mi2) 2  +  (m2l+m03)2 

2  2 

*  (m30"3mi2^m30+m12^  ^m30"hn12^  ”3^m21+m03^  ^ 

2  2 

+  (“m03+3m21) (mo3_hn2l) [ (m03+m21^  ”3^m12+m30^  ^ 


3. A  Absolute  invariant  moments 


(19) 


Combining  the  results  of  the  three  prior  subsections,  the  absolute  normalized  invariant  moments  (invariant 
to  position,  scale,  and  rotation)  are  obtained  by  replacing  m  in  (19)  with 

pq 


nPtl  (y)1+(p+q)/2 


(20) 


The  first  seven  absolute  normalized  invariant  moments  are  then: 

h  =  n20  +  f|02 


*2 

*3 

*4 


=(n20  *  n025  +  4nll 

=  (n30_3,112^  +  (3n2I  +  ''OS3 

=  (n30+n!2)  +  (n21  +  "OS3 

=  <r|30-3n12>  *n30+nl23  ^,|30+n123  '3^n21+n033  ' 

+^3n21_n033  <-,,21+r|033  ^n30+n123  —  ^ r*21+ri033  ^ 


*6  =^n20_n023  ^n30+n]23  ^n21+n033  ^ 

+  4nu(n30+n12)(n21+n03) 

•'7  =  (3n12  -n303  (n30+,’l2)  1  (n30+n12)  “3^n21+  n033  ' 

+  On2j-n03)(n2i+n03)[3(n30+ii]2)  -(n2I  +  n03)  ] 


(21) 


3.5  Case  study 

As  specific  numerical  example,  consider  an  input  square  extending  from  x^  =  1  to  x,  =  2  and  =  0  to 
y^  =  1  as  in  Figure  1.  Its  ordinary  moments  obtained  from  (6)  arc  listed  in  Table  1  .  Since  m^-  =  1,  the 

moment  mir.  and  m^,  describe  the  centroid  of  the  function.  The  central  moments  u  are  also  included  in 
1U  01  —  —  pq 

Table  1  computed  using  (12)  where  x  =  ^q/^qq  =  1*5  and  y  =  mQj/m00  =  0.5  from  (13).  Under  translation  to 

coordinates  x  =  4  to  x~  =  5  and  y^  =  0  to  y.  =  1,  the  normalized  central  moments  q  =  u  /n„ny  where 

1  2  U  1  pq  pq  UU 

y  =  l+(p+q)/2  arc  found.  The  resulting  «J>  moments  with  u  =  1  are  also  given  in  Table  1.  Comparing  those 
4>n  moments,  we  see  the  invariance  of  to  translation  of  the  input.  For  rotation  of  the  input  as  in  Figure 

3,  x  =  -0.5  and  y  =  +1.5,  the  y  values  are  shown  in  Table  1.  The  same  ;  values  again  result.  A  scale 
change  by  a  =  2  as  in  Figure  A  ?with  x  =  3,  y  =  1  nnd  =  A)  yield  identical  C’n  values  again  as  shown.  For 
this  example,  and  with  considerable  algebraic  manipulation,  the  absolute  invariance  of  the  seven  functions 
is  confirmed. 

4  .  Pj)t_ic  al  j^st_e_m  design  a_nd_  e_x_pe_r  imentjil^  demonstrat  ion 

The  coherent  optical  system  used  to  realize  (6)  is  shown  in  Figure  5.  The  simplicity  of  the  system  is  one 
of  its  prime  features.  The  Input  function  f (x,y)  is  placed  at  Pj ,  illuminated  with  coherent  laser  light,  and 
imaged  onto  a  mask  g(x,y)  at  P^.  heaving  the  light  distribution  is  fg.  hens  h^  forms  the  Fourier  trans¬ 
form  of  the  product  fg  and  on-axis  at  P  ^  we  find 

u.j(0,0)  =  ff  f  (x,y)g(x, y)dxdy  .  (22) 

If  the  mask  at  P.;  is  absent,  g  =  I  and  the  output  is  m^. 
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Fig. 3.  Rotated  Input 


Fig. 4.  Scaled  Input 


NUMERICAL  CASE  STUDT  EXAMPLE 


Table  1 


TUV.ORKT3 CAL  CASK  STUDY  (SQUARE  INPUT) 


n 

(p.q) 

1  ORIGINAL 

TRANSLATE) 

lV 

ROTATED 

(*  ) 
n 

SCALED 

<v 

m 

pq 

V 

pq 

*n 

i 

00 

i 

i 

0.166 

0.166 

0.166 

0.166 

2 

10 

1.5 

0 

0 

0 

0 

0 

3 

01 

0.5 

0 

0 

0 

0 

0 

20 

0.33 

0.083 

0 

0 

0 

0 

5 

02 

0.33 

0.083 

0 

0 

0 

0 

6 

10 

3.75 

0 

0 

0 

0 

0 

7 

03 

0.25 

0 

0 

0 

0 

0 

11 

0.75 

0 

m  =  ordinary  moments 

pq 

21 

1  .66 

0 

p  =  central  moments 

pq 

12 

0.5 

0 

nvn riant  moments 

n 

Uj(O.O) 


// 


f  (x  ,  v  )dxiiv 


in 


00* 


When  the  mask  at  is  a  simple  linearly  graded  transmittance  tune t ion  y, 
m,0  or  hiqj  respectively.  M  the  g  =  x  and  j;  =  v  masks  are  superimposed. 


u  j((U» 


t  (x,v)xv 


dxdv 


x  or  g  =  y,  the  output  at  1’ j 
=  xy  results  and  the  output  i 


Continuing  in  this  manner,  we  see  that  all  the  ordinary  moments  m  of  the  2-\)  function  f(x,v)  can  he  yv 
a  ted  with  the  appropriate  masks  >;(>:, v).  ^ 


{2\  a) 


(2  lb) 
n  e  r  - 
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SYSTEM  LENS 


Fig.  5 


Hybrid  optical/digital  system  to  compute  the 
absolute  normalized  invariant  moments. 


In  the  system  shown  in  Figure  5,  the  ordinary  moments  m  are  computed  by  the  optical  system.  The  absolute 
normalized  invariant  moments  0  are  then  calculated  from  tRSse  optically  produced  outputs  in  a  high  dynamic 
range  and  accurate  special-purpose  digital  post  processor. 

The  mask  functions  x^  y^  should  be  bi-polar.  This  can  be  achieved  by  using  a  positive  amplitude  mask  with 
a  A/2  phase  retarding  plate  in  one-half  of  the  plane.  In  our  present  experiments,  the  location  of  the  in¬ 
put  function  was  restricted  to  the  right  half  of  the  input  plane  so  that  a  unipolar  positive  mask  could  be 
used.  A  computer  controlled  cathode  ray  tube  with  10  point  correction  for  the  nonlinear  t-E  film  character¬ 
istics  was  used  to  generate  the  necessary  g=x,  g-x^,  etc.  mask  patterns  needed.  These  mask  patterns  were 
then  photographed  from  the  CRT  surface  and  inserted  in  ?2  of  the  system  shown  in  Figure  5.  Because  the  film 
masks  used  did  not  have  perfect  transmittance  t=l  or  perfect  absorption  t=0  where  necessary,  direct  computa¬ 
tion  of  m  was  not  possible,  rather  modified  m"  moments  were  produced.  To  compare  our  experimental  data, 
the  normalized  moments  m'^/m"  were  calculated^¥rom  our  measured  m"  values  and  compared  to  the  theoret¬ 
ically  obtained  ones.  The  simple  square  inputs  used  are  shown  in  Figures  6-9.  Cross-sectional  scans  of  the 

output  obtained  with  a  scanning  photometric  microscope  interfaced  to  a  high  dynamic  range  PMT  and  chart  re¬ 
corder  are  shown  in  Figure  10.  The  comparative  experimental  and  theoretical  data  are  provided  in  Table  2. 

As  seen  from  Table  2,  the  comparison  between  theory  and  experiment  is  good.  These  initial  experiments  were 
thus  most  promising. 


5.  Discussion  and  advanced  system  considerate  ms 

A  general  analysis  of  the  dynamic  range  of  the  first  ten  ordinary  moments  m  was  conducted,  from  which 

m  was  seen  to  decrease  for  higher  orders  and  to  be  inversely  proportional  to^?he  input  space  bandwidth 

p?8duct  (for  a  fixed  input  size).  For  the  case  of  a  25  mm  input  aperture  and  a  20  ey/mm  input  function,  the 

worst  case  m  (max)/m  (min)  dynamic  range  was  found  to  be  42  dB.  This  range  is  easily  satisfied  by  existing 

photo  detectBifs.  The^ciorrespond ing  dynamic  range  of  the  first  seven  invariant  moments  :  was  a  far  larger 
191  dB.  The  hybrid  optical/digital  system  shown  in  Figure  5  appears  to  be  the  best  practical  system  that 
utilizes  the  optical  system  for  the  low  accuracy  and  dynamic  range  computations  of  the  m  values  and  a 
special-purpose  digital  post  processor  for  the  high  accuracy  and  dynamic  range  computations  of  the  : ^  values. 

The  use  of  the  masks  (1+x^y^)  for  the  generation  of  the  ordinary  moments  introduces  output  bias  levels 
that  greatly  increase  the  required  photo  detector  dynamic  range  and  complicate  the  post  processing  required. 
The  necessity  for  bi-polar  masked  transmittance  functions  or  restricted  input  function  positional  lo«.it ions 
is  another  disadvantage  of  the  present  system.  Moreso,  separate  masks  arc  necessary  for  each  n  to  be  cal¬ 
culated.  We  have  thus  considered  an  alternate  system  design  in  which  all  moments  can  he  generated  in  parallel 
without  the  dynamic  range,  bias  level,  and  positive/negative  polarity  problems  associated  with  the  prior 
system.  The  concept  of  this  system  can  be  seen  by: 

(1)  replacing  x  and  y  in  (6)  by  x  cxpQ.-qX)  and  y  exp (  j  ■  )  respectively,  and 

(2)  use  of  a  Fourier  transform  rather  than  integrating  lens  and  multiple  output  detectors  rather 
than  a  single  on-axis  photo-diode  as  in  Figure  5. 


The  resultant  output  plane  and  m  equation  then  becomes: 

P<l 


P<1 


=  JJ  X!Vl  exp  |  -  )  (p-0x+q  •  gV)  ]  I  (x,  v)exp|  -  i  (  • ^X-K  y\')  Jdxdv 


t  2  -4  ) 


From  (24),  we  see  that  each  Is  now  spatial  !v  sepr. ited  at  the  output  Fourier  transform  plane  locations 


(-x-V  - 


( ->  > 


The  required  mask  is  complex.  However,  it  can  be  produced  bv  use  ol  two  real  masks  t,  =  oxp(x  cos  .^x)  and 
t  =  l+cos(x  sin  .jjX-K  |X).  After  single  sideband  filtering,  t  ^  anil  tilting  the  optical'  axis  appropr  i  at  e  1  v , 
the  filtered  complex  output  exp(jx  sin  y^x)  obtained  and  imaged  onto  t  ^  and  the  input  function  t(x,v). 
The  Fourier  transform  of  this  pattern  is  then  formed  and  (24)  results  in  the  output  plane.  All  n  values 


PH 
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are  generated  in  parallel  and  are  spatially  separated  in  the  output  transform  plane.  The  required  mask  and 
its  resolution  specifications  are  greatly  simplified  if  only  the  first  ten  ordinary  moments  m  are  required, 
as  appears  to  be  the  case.  ^ 
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Coherent  Optical  Pattern  Recognition 

DAVID  CASASENT,  fellow,  ieee 


Abstract -Pattern-recognition  systems,  techniques,  and  applications 
using  coherent  optical  systems  are  reviewed.  Many  optical  pattern- 
recognition  system  architectures  exist  that  indude  time-domain  optical 
correlators  and  the  optical  joint  transform  correlator  and  refinements 
in  the  original  optical  matched  spatial  Alter  synthesis  processor. 
Advanced  optical  pattern-recognition  tystems  are  also  described  such 
as  hybrid  optical/digital  processors  and  diffraction-pattern  sampling 
systems  using  specially  shaped  Fourier  plane  detector  arrays  The 
optical  space-variant  pattern-recognition  systems  described  are 
examples  of  the  growing  repertoire  of  operations  now  achievable  in 
optical  computers. 

I.  Introduction 

THE  BASIS  of  coherent  optical  Fourier  transform  (FT) 
theory  began  with  the  microscopy  image  resolution 
work  of  Abbe  [  1 1 ,  the  spatial-filtering  experiments  of 
Porter  [21 ,  and  the  phase-contrast  microscope  of  Zernike  [31 . 
However,  it  was  Marechal  and  Croce  [4|  in  1953  who  provided 
the  basic  analysis  of  coherent  optical  systems  in  terms  of 
spatial-frequency  response  and  thus  gave  new  meaning  to  the 
work  of  Abbe.  Significant  developments  in  optical  spatial 
filtering  began  with  the  marriage  of  communications  theory 
and  optics  in  the  classic  work  of  O’Neill  [5]  who  unified  much 
of  the  earlier  research. 

The  objective  of  the  types  of  pattern-recognition  systems  to 
be  discussed  is  to  determine  the  presence  (and  usually  the 
location)  of  a  key  object  or  signal  within  an  input  containing 
other  data  (referred  to  as  noise).  Depending  on  the  specific 
application,  the  pattern-recognition  problem  can  be  classified 
as  the  extraction  of  a  signal  buried  in  noise,  character  recog¬ 
nition  in  which  one  or  several  members  of  a  given  set  are 
searched  for,  or  the  location  of  one  key  object  in  diverse 
backgrounds.  Turin  [6]  provides  an  excellent  description  of 
matched  filters  for  such  diverse  scenarios. 

Optical  pattern  recognition,  as  one  generally  views  it  today, 
began  with  the  optical  realization  of  a  complex  matched 
spatial  filter  by  vander  Lugt  [7],  This  work  was  based  upon 
sidelooking  radar-processing  concepts  developed  by  Cutrona 
el  al.  [81  and  the  holography  work  of  Leith  and  Upatnieks 
[9|  These  early  successes  and  developments,  plus  the  com¬ 
mercial  availability  of  continuous-wave  lasers  caused  a  rapid 
maturing  in  the  field  of  optical  data  processing. 

Many  developments  have  occurred  in  the  fifteen  years  since 
the  first  optical  matched  filtering  paper.  The  intrigue  of  pro¬ 
cessing  data  in  parallel  and  in  real-time  has  captured  the 
imagination  and  tapped  the  inventiveness  of  many  researchers. 
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This  has  led  to  an  increasing  repertoire  of  coherent  optical 
pattern-recognition  systems  and  techniques.  In  this  survey 
paper,  we  provide  a  vitally  needed  summary  of  optical  pattern 
recognition  over  these  recent  fifteen  years,  and  a  consolidated 
summary  of  the  system  architectures  [10],  operations  achiev¬ 
able  [11],  and  applications  [12] -[15]  of  this  aspect  of  the 
more  general  field  known  as  optical  computing  [14] . 

Cutrona  et  al.  [16]  describe  a  wealth  of  optical  pattern- 
recognition  system  architectures  and  their  use  in  signal 
processing.  When  coded  signal  waveforms  are  used,  signal¬ 
processing  systems  become  multichannel  1  -D  versions  of  the 
more  conventional  2 -D  pattern-recognition  topologies,  and  we 
thus  include  advancements  in  such  work  in  this  survey. 

Optical  pattern  recognition  is  but  one  aspect  of  the  larger 
field  of  optical  computing  [14].  Space  does  not  permit 
detailed  discussions  of  all  optical  techniques  related  and 
associated  with  pattern  recognition.  Recent  advances  in  non¬ 
coherent  optical  pattern  recognition  are  discussed  elsewhere 
[17],  Similarly,  image  enhancement  and  restoration  tech¬ 
niques  [181  and  image  deblurring  methods  [19],  [20]  in 
which  the  output  is  a  corrected  image  are  not  discussed. 
Rather,  preference  is  given  to  systems  whose  output  contains 
the  final  desired  pattern-recognition  decision  and  data.  We 
will  concentrate  on  coherent  optical-processing  systems  in 
which  the  optical  system  itself  performs  the  actual  processing 
and  the  pattern  recognition  of  the  data,  rather  than  serving 
only  as  the  data  collection  vehicle  or  as  an  image  enhancement 
preprocessor. 

In  the  type  of  coherent  optical  processor  considered  in  this 
paper,  the  data  to  be  processed  is  assumed  to  be  an  image  or 
some  similar  two-dimensional  pattern.  This  input  data  is 
entered  into  the  optical  processor  as  a  transparency,  e.g.,  a  35- 
35-mm  slide  of  the  input  scene.  Such  a  transparent  pattern  is 
then  illuminated  with  parallel  coherent  (laser)  light  of  uniform 
amplitude  (spatially).  This  spatially  uniform  light  passes 
through  the  transparent  input  medium  and  emerges  spatially 
modulated  in  amplitude  proportional  to  the  two-dimensional 
amplitude  transmittance  pattern  recorded  on  the  input 
medium.  This  process  is  thus  quite  analogous  to  the  operation 
of  a  simple  slide-projector,  but  yet  demonstrates  the  two- 
dimensional  feature  of  such  processors. 

However,  because  the  input  illumination  used  is  coherent 
laser  light,  a  multitude  of  operations  quite  useful  in  pattern 
recognition  are  obtainable.  The  fundamental  operation  per¬ 
formed  in  such  coherent  optical  processors  is  the  Fourier 
transform  (FT)  (Section  II).  As  the  input  data  is  a  spatial  pat¬ 
tern  rather  than  temporal,  we  represent  it  in  1-f)  by  fix)  and 
in  2 -D  by  f(xt,  v , ).  Thus  in  optics,  we  speak  of  a  spatial  FT, 
in  which  distance  in  the  input  data  plane  is  directly  analogous 
to  time  in  the  more  conventional  temporal  FT.  As  we  describe 
in  subsequent  sections,  the  normal  linear-systems  operations  of 
correlation  and  convolution  (often  used  in  pattern  recogni¬ 
tion)  can  be  realized  in  such  optical  processors.  For  now 
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Eig.  I.  Schematic  of  the  conventional  optical  ET  system. 


we  only  note  that  these  operations  are  realizable  on  two- 
dimensional  input  data  and  in  parallel  (i.e.,  the  input  light 
operates  in  parallel  on  all  pixels  of  data  present  in  the  input 
plane). 

The  ability  of  an  optical  processor  to  operate  in  parallel  on 
2 -D  input  data  and  to  perform  operations  such  as  the  FT, 
correlation  and  convolution,  is  why  such  systems  have  been 
considered  for  pattern  recognition  and  other  applications. 
The  final  feature  of  such  systems  that  makes  them  attractive 
alternatives  to  digital  processors  is  the  speed  with  which  these 
parallel  operations  can  be  performed.  The  coherent  laser  light 
is  used  only  as  the  optimum  carrier  upon  which  to  modulate 
or  impress  the  input  data  and  as  the  information  bearing 
carrier  that  flows  through  the  system.  The  speed  with  which 
this  light  carrier  moves  is  roughly  1  ns/ft  (1  ns  =  10  9  s).  Thus 
for  a  three-foot  optical  system,  the  required  processing  time 
for  parallel  2 -D  pattern-recognition  operations  would  be  3  ns 
(excluding  input  data  recording  and  output  data  detection 
times).  It  is  too  premature  and  early  in  the  development  of 
optical  processors  and  not  the  purpose  of  this  paper  to  expand 
upon  the  size,  weight,  and  cost  effectiveness  of  such  systems 
compared  to  digital  or  other  alternative  analog  methods.  It  is 
also  not  the  purpose  of  this  paper  to  compare  optical  and 
digital  pattern-recognition  systems.  Rather,  our  emphasis  is  to 
convey  the  operations  achievable  and  to  summarize  recent 
progress  made  in  optical  pattern-recognition  research. 


II.  Optical  Fourier  Transeorm 
The  basic  operation  in  nearly  all  coherent  optical  systems  is 
the  FT.  It  is  well  known  [211  that  if  an  input  transparency 
with  amplitude  transmittance  f(x  x,yx),  placed  in  the  front- 
focal  plane  Px  of  a  spherical  lens  L\  (of  focal  length  fL ),  is 
illuminated  with  collimated  coherent  laser  light  (of  wavelength 
X),  then  the  light  amplitude  distribution  in  the  back-focal 
plane  P2  of  Lx  is  the  complex  2-D  FT  of  f(xx,yx).  We  repre¬ 
sent  the  2-D  optical  spatial-FT  of  f(xt,yt)  by  F(u,  v),  where 
♦  » 

F(u,u)  =  ~  JJ /(jc,,y,)exp  [j2v(uxx  +uy,)|  dxx  dyx. 

(1) 

The  schematic  of  this  simple  optical-FT  system  is  shown  in 
Fig.  1 .  We  denote  space  functions  by  lower  case  variables 
(e.g.,  f,  g,  h)  and  their  FT’s  by  the  corresponding  upper  case 
variables  (e.g.  F,  G,  H).  Distances  in  the  input  plane  Px  are 
denoted  by  (xi.yi).  The  spatial  distances  (x2,  y2)  in  the 
FT  plane  P2  are  related  to  the  spatial  frequencies  (u,  v )  present 
in  the  input  plane  Px  data  by 

k  ~x2[kfi.  v=y 2 / Xy /.  (2) 

where  the  units  of  x  and  y  are  typically  millimeter  and  the 


Fig.  2.  Example  of  the  optical  FT.  The  inserts  show  the  FT  of  the 
corresponding  regions  of  the  input  plane  />,  pattern  |  I3|.  The  direc¬ 
tion,  velocity,  and  amplitude  of  the  ocean  waves  can  he  found  from 
these  FT  data. 


units  of  u  and  v  are  cycles  per  millimeter  (analogous  to  cycles/ 
time  or  Hertz  in  the  conventional  temporal  FT). 

An  underslanding  of  the  anatomy  of  an  optical  FT  is  vital  to 
obtain  the  insight  necessary  to  follow  the  future  operations 
and  applications  to  be  discussed.  In  general,  the  transmittance 
of  P i  must  be  real  and  positive  and  thus  input  data  must  be 
recorded  on  a  bias  (the  acoustooptical  transducers  discussed 
in  Section  III  and  film-thickness  variations,  which  we  ignore, 
are  exceptions).  Thus  if  the  input  is  a  sinewave.  its  optical 
FT  at  P 2  appears  as  a  dc  spot  on-axis  plus  two  symmetrically 
located  spots  of  light  on  each  side  of  dc.  The  locations  ixj 
of  these  two  spots  correspond  to  the  input  spatial  frequency 
u  of  the  sinewave  as  in  (2).  If  the  input  sinewave  is  recorded 
horizontally  (vertically),  the  FT-plane  spots  appear  along  the 
horizontal  u  or  x2  axis  (vertical  v  or  v2  axis)  at  P2 .  If  the 
frequency  u  of  the  input  sinewave  is  increased  (decreased), 
the  separation  of  the  two  off-axis  FT-plane  spots  from  the 
origin  of  P2  increases  (decreases).  If  the  input  sinewave  is 
recorded  at  +45°  to  the  input  x,  axis,  the  spots  of  light 
comprising  the  FT-pattern  occur  on  a  line  inclined  at  45  to 
the  x2  axis  of  the  FT  plane. 

Thus  the  location  of  a  spot  of  light  in  the  optical  FT-plane 
P2  indicates  the  presence  of  a  given  spatial  frequency  in  the 
input  data,  as  well  as  the  orientation  of  this  data  in  the  input 
plane.  The  amplitude  of  each  spot  of  light  in  P2  indicates  the 
amount  of  that  spatial  frequency  component  present  in 
the  input  data.  Low  spatial  frequencies  lie  closer  to  the  origin 
of  P2. 

A  simple  example  of  the  information  content  available  in 
the  optical  FT  plane  pattern  is  shown  in  Fig.  2.  This  figure 
shows  an  aerial  photograph  of  a  portion  of  the  Caribbean  with 
the  optical  FT-pattern  of  various  regions  of  the  input  shown 
in  the  inserts  [131.  Close  examination  of  each  FT-pattern 
will  show  a  series  of  spots  of  light  at  different  orientations. 
The  direction  of  the  FT-light  pattern  indicates  the  direction  of 
the  ocean  currents  and  the  spacing  of  the  spots  indicates 
the  velocity  of  the  ocean  currents  in  the  corresponding  region 
of  the  input  scene. 

Although  the  classic  optical  pattern-recognition  system  one 
envisions  is  a  correlator  topology,  pattern  recognition  can 
often  be  achieved  by  analysis  of  the  contents  of  the  FT  plane 
alone.  When  such  a  simplified  optical  system  is  usable,  il 
is  elearly  preferable  to  the  more  sophisticated  optical  corre¬ 
lators  to  be  described  later.  The  optical  FT  pattern  is  usually 
referred  to  as  the  diffraction  pattern  Wc  now  consider  a 
diffraction-pattern  system  for  pattern  recognition  that  uses  a 
unique  frequency  plane  detector  at  P2  of  Fig.  1 
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CASASENT:  COHERENT  OPTICAL  PATTERN  RECOGNITION 

A.  Pattern  Recognition  by  Diffraction  Pattern  Sampling 

Advances  in  integrated  circuit  and  detector  technology  as 
well  as  statistical  and  nonparametric  digital  pattern-recognition 
algorithms  have  benefited  optical  pattern  recognition  just 
as  they  have  advanced  the  sophistication  of  digital  pattern 
recognition.  The  diffraction-pattern  sampling  system  to  be 
discussed  is  the  simplest  and  most  hardened  hybrid  optical/ 
digital  system.  It  is  commercially  available  from  Recognition 
Systems,  Inc.  The  key  opto-electronic  element  in  this  system 
is  a  special  Fourier  plane  detector  consisting  of  thirty-two 
wedge-shaped  and  thirty -two  ring-shaped  detector  elements  in 
the  two  semicircular  halves  of  a  one-inch  diameter  silicon 
detector  [22].  (Many  other  Fourier  plane  detectors  exist, 
and  digital-computer  systems  to  model  this  wedge/ring  detec¬ 
tor  (WRD)  have  been  described  [23],  We  restrict  discussion 
to  this  FT  plane  sampling  system  as  representative  of  such 
pattern-recognition  processors.)  The  outputs  from  all  sixty- 
four  elements  in  this  WRD  are  available  in  parallel  and  can  be 
multiplexed,  electronically  preprocessed,  and  fed  to  a  digital 
computer  for  final  analysis. 

The  detector  is  placed  in  the  FT  plane  P2  of  the  optical 
system  of  Fig.  1 .  The  outputs  of  the  ring-shaped  detector 
elements  provide  the  analyst  with  information  on  the  spatial 
frequencies  present  in  the  input  data,  on  the  amount  of  each 
spatial  frequency  present,  and  thus  a  measure  of  the  informa¬ 
tion  content  of  the  input  data.  The  outputs  of  the  wedge 
detector  elements  provide  data  on  the  orientation  of  the 
information  in  the  input  plane. 

When  the  WRD  is  used  in  a  pattern-recognition  application, 
the  outputs  of  only  selected  wedge  or  ring  detector  elements 
are  used,  thus  resulting  in  a  large  reduction  in  data  analysis. 
However,  to  determine  which  WRD  outputs  are  to  be  used  and 
how  sums  or  differences  of  selected  detector  outputs  are  to  be 
weighted,  a  test  set  of  inputs  is  used.  All  WRD  outputs  are 
collected  for  all  of  these  test  inputs.  Statistical  tests  are  then 
applied  to  determine  if  these  data  are  separable  and  if  recogni¬ 
tion  algorithms  can  be  developed.  Nonparametric  pattern- 
recognition  methods  are  then  developed  for  these  data.  Once 
the  recognition  routine  has  been  determined  analysis  of  future 
inputs  is  automatic  and  rapid. 

This  type  of  optical  pattern  recognition  is  useful  in  applica¬ 
tions  requiring  coarse  sorting  and  the  analysis  of  large  amounts 
of  data.  When  such  a  method  is  appropriate,  a  system  using 
WRD  data  is  faster  and  cheaper  than  an  optical  correlator 
because  it  requires  the  analysis  of  far  less  data  than  that  con¬ 
tained  in  the  entire  input  (usually  only  five  or  fewer  ring 
measurements  are  needed  in  the  final  system).  If  all  points 
in  the  input  plane  pattern  are  needed  to  sort  the  data,  matched 
spatial  filtering  or  other  correlation  pattern-recognition 
methods  are  required.  However,  if  the  resolution  of  the  input 
data  is  finer  than  what  is  needed  and  if  the  input  pattern  is 
large  and  generally  uninteresting  except  for  certain  smaller  key 
regions,  then  diffraction  pattern  sampling  using  the  WRD  at 
P2  of  Fig.  1  is  an  appropriate  pattern-recognition  technique. 

Many  uses  of  the  WRD  diffraction  pattern-sampling  system 
have  been  demonstrated  [22] .  These  include:  distinguishing 
urban  from  nonurban  imagery  [24]  (by  the  higher  spatial 
frequency  data  present  in  urban  imagery  at  higher  rings); 
distinguishing  various  physiographic  regions  of  an  image  [25] ; 
and  analysis  of  X-ray  images  to  determine  the  presence  of 
black  lung  disease  [26).  In  this  last  case,  textural  image  data 
is  used.  Sixty-two  features  of  each  of  six  lung  zones  are 
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Eig.  3.  Schematic  of  a  real-time  spatial  integrating  I  -D  optical  correlator. 

measured,  feature  extraction  is  achieved  by  a  step-wise  dis¬ 
criminant  analysis,  and  a  correct  classification  of  over  86  per¬ 
cent  of  the  X-rays  examined  was  achieved  using  the  data 
from  only  two  to  five  ring  detectors.  Detection  of  muscular 
dystrophy,  the  screening  of  cervical  cytologic  samples  for 
malignancy,  a  flow  cell  analysis  system,  measurement  of  line- 
widths  in  photomasks,  the  inspection  of  griege  goods,  the 
determination  of  the  printability  of  paper  and  the  analysis  of 
handwriting  are  among  other  more  well-known  laboratory  and 
factory  applications  of  this  WRD-system  [22] . 

The  WRD-system  is  best  suited  for  more  general  pattern- 
analysis  applications.  The  recognition  of  specific  image  fea¬ 
tures  generally  requires  the  use  of  more  sophisticated  pattern- 
recognition  methods.  This  is  due  to  the  loss  of  resolution 
and  the  phase  portion  of  the  FT  that  occurs  when  only  the 
diffraction  pattern  is  sampled.  Hereafter,  we  discuss  only 
optical  pattern-recognition  systems  that  achieve  correlation 
and  matched  spatial  filtering.  These  systems  are  more  power¬ 
ful,  but  are  more  complex  and  less  extensively  available 
commercially . 

III.  Time-Domain  Optical  Pattern  Recognition 

Correlation  is  still  the  basic  optical  pattern-recognition 
operation.  In  1-D,  we  describe  the  correlation  of  two  time 
functions  or  signals  by 

P(T)  =  f  g(t)  h(t +  r)  dt  =  g  ®  h  (3) 

where  g  is  the  input  function,  h  is  the  reference  function,  and 
T  is  the  shift  parameter.  To  realize  the  parallel  processing 
advantages  of  an  optical  processor  in  real  time,  transducers 
capable  of  introducing  the  input  data  into  the  system  in  real 
time  are  needed.  The  most  available  and  proven  real-time 
optical  transducers  are  acoustooptic  cells  [27],  In  these 
devices,  the  input  signal  is  fed  to  a  transducer  on  the  acousto¬ 
optic  cell  and  a  sound  wave  proportional  to  the  input  signal 
travels  along  the  cell.  Korpel,  Minkoff,  King,  and  Lambert 
et  al.  pioneered  much  of  the  research  and  applications  of  these 
devices,  especially  in  radar  processing  [  28 ]  — [  3 1  ] .  These 
systems  usually  realize  the  correlation  operation  in  the  time 
domain  and  thus  we  discuss  them  separately  here.  These 
acoustooptic  cells  are  1-D  transducers  and  thus  the  resultant 
correlators  are  only  1-D  and  hence  are  generally  restricted  to 
signal  pattern-recognition  applications.  Because  of  the  large 
center  frequencies  (up  to  I  GHz)  and  bandwidths  (up  to 
500  MHz)  of  available  acoustooptic  cells,  such  systems  are  of 
direct  use  in  radar  signal  pattern  recognition.  Many  of  these 
concepts  originated  elsewhere  and  are  reviewed  in  [32] . 

The  schematic  of  a  spatial  integrating  1-D  correlator  is 
shown  in  Fig.  3.  The  operation  of  these  systems  is  straight¬ 
forward.  The  coded  reference  signal  h(t)  is  stored  at  Flt)  as 
1  +  h(Xi).  The  real-time  received  input  signal  gft)  is  applied  to 
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Fig.  4.  Schematic  of  a  time  integrating  real-time  1  -D  optical  correlator. 

the  transducer  at  Pia.  As  this  signal  moves  through  the  input 
P |fl  aperture,  it  causes  a  spatial  variation  in  the  index  of  refrac¬ 
tion  of  the  cell.  This  results  in  a  signal  in  Pla  that  varies  in 
space  (distance  along  the  cell)  and  time  t.  We  thus  repre¬ 
sent  this  signal  as  g(xi  +0*0,  where  vs  is  the  velocity  of  the 
sound  wave  which  moves  vertically  in  the  x ,  direction  in 
Fig.  3.  The  wavefront  leaving  Pia  is  phase-modulated  by 
g(Xj  +  vst)  and  is  described  by 


incident  on  the  output  detector  array  is  now 

Uj(x,  l)  =  |fl,  +  g</)|  |fij  +  h(x  +  Pjf)| .  (8) 

In  this  topology,  integration  of  the  product  in  (8)  is  per¬ 
formed  on  the  output  detector  array  and  the  dc  bias  terms  are 
removed  by  ac  coupling.  Once  again  the  desired  correlati  m  of 
g  and  h  results.  In  this  system,  the  correlation  is  performed  in 
time  and  displayed  in  space.  We  thus  refer  to  this  as  a  time- 
integrating  correlator. 

Extensions  of  these  1  -D  systems  to  multichannel  correlators 
and  the  use  of  heterodyne  detection  to  yield  complex  correla¬ 
tions  are  among  the  many  extensions  possible  in  these  time- 
domain  pattern-recognition  systems  1321.  The  interaction  of 
bulk  laser  light  and  surface  acoustic  waves  represents  another 
viable  1  -D  optical  signal  processing  approach  that  promises 
small  size  systems  of  high  bandwidth  (35) . 

IV.  Frequency-Domain  Optical 
Pattern  Recognition 


r)  =  a0  exp  (/cof)  exp  [jbg(x,  +usf)l  (4) 

where  b  is  a  constant  and  a0  exp  (/cor)  describes  the  plane- 
wave  input  of  uniform  amplitude  a0  traveling  in  time.  The 
Lla  and  Lxb  imaging  system  and  slit  filter  convert  the  phase 
modulation  in  (4)  to  amplitude  modulation  by  passing  only 
the  first-order  term  in  the  transform  of  (4).  The  filtered  image 
of  t\  incident  on  Pib  is  then 

r)  =  jba0g(xi  +  vst)  exp  (/cot)  (S) 

and  the  light  distribution  transmitted  through  Pxb  is  fill  + 
h(x  i )] . 

Lens  Lj  produces  the  FT  of  this  product  of  two  filtered 
signals  at  P3 ,  where  the  FT  is  evaluated  only  on-axis  by  a 
single  photodetector  at  u  =  0.  1.2  thus  effectively  integrates 
f'i(l  +h(xi)]  over  a  spatial  distance  vsT  where  T  is  the  time 
aperture  or  time  window  of  the  input  acoustooptic  cell.  The 
time  history  of  the  output  of  the  photodiode  at  P3  is  thus 
(omitting  terms  with  signal  functions  with  no  Fourier  compo¬ 
nents  with  periods  of  the  order  of  vsT  or  larger). 


u3  (/)  = 


* 


g(x  +  ust)  h(x)  dx 


or  the  magnitude  squared  of  the  desired  correlation  of  the 
input  and  reference  signals.  Since  this  correlation  is  performed 
in  space  and  displayed  in  time,  the  name  spatial-integrating 
correlator  is  used.  With  the  correlation  shift  r  between  the 
two  functions  obtained  by  the  movement  of  g(t)  through  the 
acoustooptic  cell,  this  and  similar  systems  using  moving- 
window  transducers  are  time-domain  correlators. 

A  time-integrating  version  of  this  correlator  (331  is  shown 
in  Fig.  4.  This  topology  is  similar  to  a  noncoherent  processor 
described  earlier  by  Bromley  [17]  and  later  extended  by 
Goodman  (34|  for  use  with  a  linear  input  LED  array  for 
matrix-vector  multiplication  applications.  In  this  system, 
the  time-output  intensity  from  an  LED  is  modulated  by  the 
input  signal  g(t)  and  the  input  light  is  now 


The  architecture  of  the  correlators  described  in  Section  III 
was  determined  by  the  real-time  acoustooptic  input  transducer 
used  in  which  the  signal  moved  through  the  input  plane. 
Although  permitting  processing  of  high-bandwidth  input 
data,  the  space  or  time  bandwidth  product  of  acoustooptic 
cells  is  limited  to  2000  and  the  time  apertures  are  typically 
limited  to  10  /is.  In  addition,  these  systems  are  basically  1-/7 
correlators  and  more  conducive  for  signal  rather  than  image 
pattern  recognition.  Optical  systems  with  a  fixed  rather  than 
moving  input  data  frame  and  systems  that  achieve  correlation 
by  multiplication  in  the  frequency  domain  rather  than  the 
time  domain  or  space  domain  are  more  customarily  used  in 
image  and  2-/7  pattern  recognition . 

The  basic  equations  from  linear-system  theory  that  describe 
such  correlators  are: 

p{x,y)  ~g(i,  i?)  ®  h(£,  t?)  =  correlation 


oo 

’If 


g($,V)h*(x  +£,y  +  rj)  d£  drj 


=  5'1[G(u,u)//*(u,u)l  (9) 

r(x,y)  =  £(1;,  tj)  *  h({j,  tj)  =  convolution 


IT* 


g($,n)h(x  -  n)  </£</») 

=  ‘.r'\G(u,v)H(u,v)].  (10) 


We  represent  the  inverse  FT  by  and  the  forward  FT  by  5. 
Optically  these  two  operations  are  essentially  equivalent  if  the 
output  coordinate  axes  are  properly  defined.  From  the  last 
formulations  in  (9)  and  (10),  we  see  that  the  convolution  of 
two  functions  can  be  produced  by  forming  the  FT  of  the 
product  of  the  FT’s  of  the  two  functions.  When  the  conjugate 
transform  of  the  reference  function  is  used  (a  superscript  * 
denotes  complex  conjugate),  the  correlation  results.  Several 
2-Z7  optical  correlator  architectures  are  now  described  and 
examples  of  the  outputs  of  such  systems  are  presented. 


r,(t)  =  Bx  +g(t)  (7) 

where  fl,  is  the  input  bias  level.  The  reference  signal  is  added 
to  a  bias  and  used  1°  amplitude  modulate  the  center  carrier 
frequency  of  an  acoustooptic  cell.  The  light  distribution 


A.  Frequency  Plane  Correlator 
The  optical  system  of  Fig.  5  directly  implements  (9).  With 
g(xt,  y i )  placed  at  P i,  the  light  distribution  incident  on  P2  is 
(l(u.  u).  With  the  transmittance  of  P2  described  by  H*(u ,  u). 


CASASENT:  COHERENT  OPTICAL  PATTERN  RECOGNITION 


hV»  «»h 

Fig.  5.  Schematic  of  a  frequency  plane  correlator  (FPC)  with  a  matched 
spatial  filter  (MSF)  of  the  reference  function  recorded  at  Pt . 


THE  DCVEL0IA4ENT  Of  RAOAR  DURING  WRLD 

dm  ii  brought  raoar  im  a  laboratory 

CONCEPT  TO  A  MATURE  DISCIPLINE  IN  JUST 
A  FEM  SHORT  TEARS.  SINCE  1945  RADAR 
TECmOLOGY  HAS  BECOME  SO  SORHISTICATEO 
THAT  THE  SASIC  RECTANGULAR  IAILSE  RADAR 
SIGNAL  IS  NO  LONGER  SUFFICIENT  IN  THE 
OCSIW  OF  MANY  NEM  RADAR  SYSTEMS. 

MORE  COMPLEX  RADAR  SIGNALS  MUST  BE 
TAILORED  TO  SPECIFIC  REQUIREMENTS. 

(a) 


(b) 


Fig.  6.  Representative  example  of  optical  pattern  recognition  using  the 
system  of  Fig.  5.  (a)  Input  plane  F,  pattern,  tb)  Output  correlation 
plane  P 3  pattern  with  a  matched  spatial  filter  of  the  word  “RADAR” 
recorded  at  F,  of  Fig.  5.  The  location  of  the  correlation  peaks  cor¬ 
respond  to  the  locations  of  the  word  “RADAR"  in  the  input  para¬ 
graph  of  text  |  36 1 . 

the  light  distribution  leaving  P2  is  G(u,  v)  H*(u ,  e).  Lens  1.2 
forms  the  FT  of  this  product  G(u ,  u)H*(u,v )  of  two  trans¬ 
forms  as  described  by  (9).  Thus  the  desired  correlation  of 
g  and  h  results  at  P3  of  Fig.  5.  The  display  of  an  optical- 
correlation  plane  differs  significantly  from  a  digital  correlation 
with  the  coordinates  of  P3  being  the  (x,y)  shift  parameters  of 
the  correlation.  The  optical-correlation  pattern  in  P3  consists 
of  peaks  of  light.  The  presence  of  a  peak  of  light  in  P3 
indicates  the  presence  of  h  in  g,  the  location  of  the  peak  of 
light  denotes  the  location  of  h  in  g,  and  the  amplitude  of  the 
peak  denotes  the  degree  to  which  h  and  the  associated  region 
of  the  input  g  agree.  This  coding  of  the  locations  and  ampli¬ 
tudes  of  the  peaks  of  light  at  P3  in  an  optical-output  correla¬ 
tion  plane  are  best  shown  by  example.  In  Fig.  6  we  show  the 
input  plane  pattern  (a  paragraph  of  text  =  g )  and  the  resultant 
output  correlation  plane  pattern  (with  H*  being  the  conjugate 
transform  of  the  word  “RADAR”).  The  reader  can  convince 
himself  that  the  locations  of  the  peaks  of  light  in  Fig.  6(b) 
correspond  to  the  locations  of  the  word  “RADAR”  in  the 
input  paragraph  of  text. 

We  can  describe  the  example  of  Fig.  6  mathematically  with 
increased  insight  into  the  process  that  occurs.  To  achieve 
such  a  formulation,  we  describe  the  input  by 

g{x,y)  =  f(x,y)  +  £  h(x  -  x„,y  -  y„)  (11) 

»»i 

where  h  represents  the  word  “RADAR,”  ( xn,y„ )  denote  the 
six  locations  of  this  word  in  the  full  input  g,  and  f  describes 


Si  7 

the  remaining  portions  of  the  full  input.  The  transform  of 
(11)  incident  on  P2  is 

6 

G(u,i>)  =  F+£  H exp  l~j2n(uxn  +  vyn)\ .  (12) 

n  =  1 

The  transmittance  of  P2  is  H*  and  thus  the  light  distribution 
leaving  P2  is 

U2(u,  v)  =  GH*  =  FH*  +  £  HH»  exp[-j2it(ux„+vy„)\. 

n=l 

(13) 


=  T®  h  +  h  ®  h  *  8(x  -  x n,y  -  yn) 

n  =  1 

=  £  5(x  -  x„,y  -  y„).  (14) 

pi =i 

In  the  last  expression  in  (!4),  the  autocorrelation  h  ®  h  of  h 
was  assumed  to  be  a  delta  1  unction  of  normalized  unit  ampli¬ 
tude  (i.e.  h  ®  h  =  1 )  and  the  key  object  and  the  other  regions 
of  the  input  are  assumed  to  be  uncorrelated  (i.e.,  f  ®/i  =  0). 
The  resultant  correlation  of  g  and  h  thus  reduces  to  six  delta 
functions  at  locations  (x„,y„)  corresponding  to  the  six  loca¬ 
tions  of  h  in  g. 

The  correlator  of  Fig.  5  is  the  classic  matched  spatial  filter 
(MSF)  system  devised  by  Vander  Lugt  [7]  in  1964.  The  H* 
pattern  recorded  at  P2  is  an  MSF.  We  refer  to  this  as  a  fre¬ 
quency  plane  correlator  (FPC)  since  the  desired  correlation  is 
achieved  by  multiplying  the  FT’s  of  g  and  h.  One  of  Vander 
Lugt's  major  contributions  was  the  description  of  how  a  filter 
function  with  complex  transmittance  H*  could  be  recorded 
on  an  intensity-sensitive  medium  at  P2.  The  scheme  used  is 
similar  to  the  holographic-synthesis  procedure  originated  by 
Gabor  |37],  and  developed  by  Leith  and  Upatnieks  (91  for 
3-D  imaging  applications. 

B.  Matched  Spatial  Filter 

The  MSF  synthesis  procedure  as  described  by  Vander  Lugt 
assumed  g  to  be  the  signal-plus-noise  and  the  application 
directly  envisioned  was  the  recovery  of  a  signal  h  buried  in 
noise.  In  our  pattern-recognition  application,  those  portions 
of  the  input  that  do  not  agree  with  the  signal  or  reference 
function  h  are  effectively  the  noise  or  the  function  /‘in  (11). 
To  record  H*  at  P2 ,  the  function  h  is  placed  at  F,  and  the 
interference  of  its  transform  H  (formed  by  Lt  at  P2)  and  a 
plane  wave  reference  beam  Ur  =  exp  (/2trax2 )  of  unit  ampli¬ 
tude  is  recorded  at  P2,  where  or  =  (sin  0)/X  is  the  spatial  fre¬ 
quency  due  to  the  off-axis  angle  6  between  the  reference  and 
signal  beams.  The  pattern  recorded  at  P2  and  the  subsequent 
transmittance  of  P2  is 

t3(x2,y2)  =  \m-UR  |2  =  1  +|//jJ 

+  //  exp  (/2traxj)  +  H*  exp  (-j2nax2).  (15) 

The  last  term  in  (15)  is  proportional  to  the  desired  transmit¬ 
tance  H*  of  P2.  To  see  how  the  effects  of  this  term  are 
separated  from  the  other  three  terms  in  (15),  we  consider  the 
correlation  process  itself  in  detail. 


Lens  L2  forms  the  FT  of  U2  and  at  P3  we  obtain 
u3(x,y)  =  JlU2]  =‘JIGN*} 
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During  correlation,  g  is  placed  at  Px  and  the  full  transmit¬ 
tance  of  Pi  is  ti  given  by  (15).  The  light  distribution  leaving 
Pi  is  Gti  and  L2  of  focal  length  fL2  forms  the  FT  of  this 
product  at  P3  ,  where  we  find 

u3(x3,y3)  =  ‘J[Gt2\  =  g  atx3  =  y3  =0 
+  h  ®  h  *  g  at  x3  =y3  =  0 
+  g  *  h  at  x3  =  a\fL2,y3  =  0 
+  g  ®  h  at  x3  =  -a\fL2,y3  =  0.  (16) 

From  (16),  we  see  that  the  desired  correlation  g  ©  h  can  be 
separated  from  the  other  three  terms  in  the  output  plane 
pattern  by  proper  choice  of  the  reference  to  signal -beam  angle 
6.  The  region  of  the  correlation  plane  shown  in  Fig.  6  is  the 
portion  centered  at  jc3  =  ~ofKfi2,y3  -  0. 

The  FPC  optical  pattern-recognition  system  of  Fig.  5  is 
useful  when  a  single  fixed  reference  function  is  to  be  located. 
In  many  applications  (i.e.,  character  recognition),  multiple 
and  composite  MSF’s  are  useful  to  allow  a  parallel  pattern- 
recognition  search  to  be  performed.  Several  multiple  MSF 
approaches  are  summarized  in  [381 .  Much  of  the  work  in 
this  area  relates  to  optical  data  storage  [39] .  Optical  pattern- 
recognition  techniques  have  been  applied  to  a  multitude  of 
applications  including  diatom  (water  pollution  particle)  [40], 
text  [41],  aerial  imagery  [42],  and  key-object  [43]  recogni¬ 
tion.  Single,  multiple,  and  averaged  MSF’s  have  been  used  in 
[40]  and  [43]  to  reduce  the  number  of  required  filters  and 
to  enchance  the  capacity  of  the  recognition  system. 

C.  Joint  Transform  Correlator 
An  alternate  optical  pattern-recognition  topology  [44]  is 
shown  in  Fig.  7.  In  this  architecture,  the  two  functions  to  be 
correlated  (g  and  h,  assumed  to  be  of  spatial  extent  b)  are 
placed  side  by  side  at  P\  with  a  center-to-center  separation  7b. 
The  transmittance  of  P\  (in  1  -D,  for  simplicity  only)  is 

u(xl)=g(xt  -  b)  +  h(xi  +b).  (17) 

Lens  L\  forms  the  FT  of  (17)  or  a  joint  transform  pattern 
that  is  recorded  at  P2 .  The  subsequent  transmittance  of  P2  is 

t2(u)  =  | G  exp  (-j7nub)  +  H  exp  (+j2itub)\2 

=  |C|2  +  \h\2  +  GH*  exp  (~j4vub)  +  G*H  exp  (+j4irub). 

(18) 

P2  is  now  illuminated  with  a  plane  wave  of  read-light  (shown 
incident  on  P2  in  reflection  in  Fig.  7). 

With  P2  in  the  front  focal  plane  of  L2,  the  pattern  at 
P  is  the  FT  of  (18)  or 

U3(*3)  =  £®£  atx-.  =0 

+  h  ®  h  at  x3  =0 

+  g  ®  h  at  x3  =  +2 b 

+  h  ®  g  at  x3  =  -2b.  (19) 

Proper  choice  of  the  separation  2b  between  the  inputs  in 
Fig.  7  enables  the  desired  g  ®  h  term  in  the  output  pattern 
of  (19)  to  be  separated  from  the  other  terms.  This  is  similar 
to  how  the  choice  of  0  in  the  FPC  system  of  Fig.  5  enabled 
the  desired  correlation  term  to  be  separated  from  other  terms 
in  that  output  plane.  This  is  referred  to  as  a  joint  transform 
correlator  (JTC)  [44] . 
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Fig.  7.  Schematic  of  a  joint  transform  correlator  (JTC)  for  optical 
pattern  recognition. 

D.  Real-Time  Optical  Pattern  Recognition 

Since  the  advantages  of  optical  pattern  recognition  include 
real-time  and  parallel  processing,  reusable  and  real-time  2-D 
electrooptical  transducers  are  necessary  at  the  input  plane/5!, 
and  often  at  the  transform  plane  P2  in  the  systems  of  Figs.  5 
and  7,  if  real-time  operation  is  to  be  realized.  The  devices 
used  for  these  purposes  are  known  as  spatial  light  modulators 
(SLM)  [45],  [46].  The  architecture  of  the  optical  pattern- 
recognition  system  of  Fig.  7  assumed  an  optically  addressed 
and  reflex-mode  readout  SLM  at  P2  with  different  write  and 
read  wavelengths.  Most  optically  sensitive  SLM’s  operate  in 
this  manner. 

The  example  shown  in  Fig.  6  was  performed  in  real-time  on 
the  FPC  system  of  Fig.  5  using  an  electron-beam  addressed 
electrooptic  crystal  input  plane  Px  transducer.  The  input 
paragraph  of  text  was  scanned  onto  a  special  target  crystal 
by  a  current-modulated  electron  gun,  quite  similar  to  the 
operation  of  a  television.  A  collimated  laser  beam  was  then 
passed  through  this  crystal  and  emerged  spatially  modulated 
with  the  input  data.  The  electrooptic  effect  [47]  is  used 
in  this  and  similar  devices  to  modulate  an  input  light  wave 
proportional  to  the  spatially  varying  voltage  or  charge  pattern 
across  the  crystalline  material  [45] ,  [46] . 

Another  real-time  optical  pattern-recognition  application 
nearing  completion  at  General  Motors  [48]  is  shown  in  Fig.  8. 
The  application  involved  the  recognition  and  location  of  the 
position  of  a  3 -D  object  (a  relay  for  the  example  shown  in 
Fig.  8)  on  an  assembly  line.  The  output  of  this  optical  pattern- 
recognition  system  is  to  be  interfaced  to  a  robot  as  part  of  an 
automated  assembly  line.  In  operation,  a  diffusely  illuminated 
image  of  the  assembly  line  is  continuously  focused  onto  the 
photoconductor  side  of  an  SLM  at  Px  of  Fig.  5.  The  image  on 
the  liquid-crystal  input  transducer  used  is  read  out  continu¬ 
ously  in  reflection  in  coherent  laser  light.  Since  the  decay 
time  of  the  liquid-crystal  SLM  is  only  20  ms.  the  liquid-crystal 
input  transducer  serves  as  a  noncoherent-to-coherent  input 
Px  image  converter  for  the  FPC  system  of  Fig.  5.  Several 
images  of  the  input  object  (a  relay)  on  the  assembly  line  are 
shown  in  Fig.  8(b).  An  MSF  of  the  key  object  to  be  searched 
for  (the  relay)  is  stored  on  film  at  P2  of  Fig.  5.  The  peak  in 
the  output  correlation  plane  P3  pattern  and  its  cross  sectional 
scan  are  shown  in  Fig.  8(a)  for  several  locations  of  the  input 
relay  on  the  moving  assembly  line  in  real  time. 

To  change  the  key  object  to  be  searched  for,  only  the  film 
at  P2  need  be  changed.  A  modified  scan  and  prism  system  is 
presently  under  consideration  at  General  Motors  to  enable 
recognition  of  a  key  object  to  be  maintained  in  the  presence 
of  expected  rotational  and  scale  differences  between  the  input 
and  reference  (or  MSF)  imagery.  In  Sections  V  and  VI.  we 
consider  alternate  methods  by  which  such  distortion  invari¬ 
ance  and  increased  optical  pattern-recognition  flexibility  can 
be  realized. 
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Fig.  8.  (a)  Example  of  a  real-time  optical  pattern  recognition  (48).  (b) 
Output  correlation  peak  and  its  cross  sectional  scan  for  various  loca¬ 
tions  of  the  input  object  (a  relay)  in  the  input  field  of  view. 


An  alternate  real-time  2-D  optical  pattern-recognition  system 
[491  intended  for  signal-processing  applications,  specifically 
the  generation  of  the  ambiguity  function  (range  versus  Dopp¬ 
ler)  for  any  coded  input  waveform,  is  shown  in  block  diagram 
form  in  Fig.  9.  This  topology  combines  the  high  bandwidth 
features  of  the  acoustooptic  correlators  described  in  Section 
III  with  the  2-D  nature  of  an  optical  processor.  In  signal 
correlations,  coded  waveforms  are  generally  employed  and  the 
target’s  range  and  Doppler  vary.  The  desired  target  data  (its 
ranRe  and  Doppler)  is  best  conveyed  by  an  ambiguity  surface 
(50)  with  range  r  and  Doppler  v  axes.  This  output  can  be 
realized  in  a  multichannel  correlator  in  which  different 
Doppler  shifted  versions  of  the  reference  signal  are  recorded 
on  different  lines.  Each  of  these  signals  is  then  correlated 
with  the  received  signal  in  1  -D.  The  resultant  2-D  plot  (cor¬ 
relation  or  range  t  horizontal  and  Doppler  vertical)  is  an 
ambiguity  function,  widely  used  in  radar.  This  basic  signal 
pattern-recognition  problem  thus  deserves  note  in  this  paper. 
Many  methods  exist  by  which  this  operation  can  be  realized. 
One  of  the  more  promising  and  developed  ones  is  shown 
schematically  in  Fig.  9  and  is  described  below. 

In  the  pulse  compression  section  (Fig.  9(a)),  the  coded  radar 
waveform  to  be  used  is  generated  by  computer  or  other 
methods  and  displayed  on  a  CRT  from  which  it  is  continu¬ 
ously  imaged  onto  a  liquid  crystal.  The  received  input  radar 
signal  is  fed  to  an  acoustooptic  input  transducer.  This  received 
input  signal  is  continuously  imaged  onto  the  reference  signal 
(on  the  liquid  crystal)  in  reflection,  the  product  integrated 
and  heterodyne  detected  by  a  photodiode,  whose  time-history 
output  is  the  correlation  of  the  transmitted  coded  radar  signal 
and  the  reflected  signal  received  from  various  targets  for 
various  transmitted  pulses  in  the  pulse  burst.  If  this  process 
is  continued  for  IV  pulses  in  a  pulse  burst  waveform,  N  output 
correlations  appear  in  time  from  the  photodiode.  These  out¬ 
puts  are  separated  in  time  by  one  pulse  period  and  correspond 
to  the  returns  from  targets  at  a  specific  range,  with  each  of 
the  At-retums  modulated  by  the  target’s  Doppler.  If  these 
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Fig.  9.  Schematic  of  a  2-D  real-time  optical  signal  processor,  (a)  Pulse 
compression  section,  (b)  Doppler  processing  section  (49|. 


correlations,  corresponding  to  the  returns  due  to  sequential 
transmitted  pulses,  are  recorded  on  separate  horizontal  lines, 
they  will  align  vertically  for  a  target  at  a  given  range. 

To  obtain  Doppler  data  on  the  target,  one  must  integrate 
across  these  N  correlations.  To  achieve  this,  the  output  from 
the  photodetector  is  fed  to  a  second  acoustooptic  cell  (Fig. 
9(b).  A  laser  is  pulsed  on  in  synchronization  with  the  PRF  of 
the  code  and  passed  through  a  1  -D  deflector.  In  the  resultant 
pattern  recorded  on  the  second  liquid  crystal,  each  of  the  N 
correlations  from  the  pulse  compression  section  are  recorded 
on  N  separate  lines.  The  vertical  scanning  system  for  this  2-D 
pattern  is  synchronized  to  the  PRF  of  the  transmitted  wave¬ 
form.  This  second  liquid  crystal  is  then  read  from  the  right 
side  and  the  1  -D  vertical  FT  of  the  pattern  on  this  liquid 
crystal  yields  the  desired  range/Doppler  ambiguity  function. 
This  system  is  scheduled  for  completion  and  delivery  in  late 
1978. 


V.  Weighted  MSF  Synthesis 
As  implied  in  Section  IV,  practical  pattern-recognition  sys¬ 
tems  of  any  kind  must  be  able  to  operate  even  with  various  dis¬ 
tortions  (scale,  rotation,  aspect, etc.)  present  between  the  input 
and  reference  functions.  Such  flexibility  is  achieved  in  digital 
pattern  recognition  by  sophisticated  software  algorithms. 
Recent  programs  intended  to  increase  the  flexibility  of  optical 
pattern-recognition  systems  are  addressing  such  practical 
problems  with  encouraging  results.  One  approach  toward  this 
goal  involves  control  of  the  MSF  synthesis  process  (42).  To 
describe  this  method,  we  rewrite  (1 5)  as 

h(x2>yi)  =  1  +  1/AT  +  (2I\/K)  cos  (20) 

where  K  =  /t2/|/f|5  is  the  ratio  of  the  intensities  of  the  refer- 
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ence  and  signal  beams  an<J  where  \p  =  2 nax2  +  Arg  (H) 
describes  the  recorded  MSF  as  a  fringe  pattern  at  spatial  fre¬ 
quency  a  with  the  phase  of  H  encoded  as  a  phase  modulation 
on  this  carrier  frequency  a. 

At  this  point,  it  is  necessary  to  discuss  one  feature  of  all 
MSF  optical  pattern-recognition  systems.  We  refer  to  the 
limited  dynamic  range  of  the  P2  recording  material.  The 
optical  FT  of  an  image  contains  a  large  dc  term  (that  con¬ 
tains  no  information  useful  in  distinguishing  objects)  and 
higher  frequency  components  of  far  lower  intensity  (that 
contain  the  data  needed  to  distinguish  or  discriminate  one 
object  from  another  similar  one).  The  bias  on  the  MSF 
material  is  properly  adjusted  so  that  this  dc  term  saturates  the 
MSF  material,  whereas  high  modulation  is  given  to  the  lower 
intensity  and  higher  spatial  frequency  data.  Thus  in  practice, 
an  optical  FPC  system  correlates  the  ac  rather  than  the  dc 
portions  of  imagery. 

Referring  to  (20),  we  see  that  since  H  varies  spatially  (with 
x2  and  u)  so  does  K  and  hence,  so  does  the  modulation  of  the 
fringe  pattern.  Recently  [42]  use  has  been  made  of  this  fact 
to  choose  the  spatial  frequency  /*  at  which  K  =  1  (corre¬ 
sponding  to  full  modulation).  This  effectively  weights  specific 
spatial  frequencies  in  the  MSF.  The  use  of  this  adjunct  to 
optical  pattern  recognition  is  best  shown  by  example.  The 
image  of  Fig.  10(a),  characterized  by  regions  A,  B ,  and  C  that 
are  respectively  predominantly  rural,  urban  and  structured 
was  used  as  the  test  image.  An  MSF  of  the  full  image  was 
made  with  K  =  2  at  various  spatial  frequencies  /*  and  corre¬ 
lated  with  the  three  indicated  isolated  regions  of  Fig.  10(a). 
In  Fig.  10(b)  we  show  the  variation  of  the  peak  intensity 
Ip  of  these  correlations  with  the  /*  band  in  which  K  -  1 
during  MSF  synthesis.  The  correlation  for  the  urban  input 
(curve  B )  clearly  peaks  at  a  larger  /*  setting  than  for  the 
rural  input  (curve  A),  with  the  correlation  for  the  structured 
image  region  (curve  C)  peaking  at  an  intermediate  /*  setting. 
From  this  example,  we  clearly  see  that  control  of  K  and  /*  can 
greatly  effect  the  resultant  correlation.  The  WRD  detector 
(Section  II-A)  was  used  to  facilitate  rapid  evaluation  of  the 
effects  of  f*  and  K  settings  on  the  output  correlation  [42  J . 

This  same  technique,  which  we  refer  to  as  weighted  MSF 
synthesis,  is  also  useful  in  controlling  the  effect  of  various 
expected  image-degradations  on  Ip  and  signal-to-noise  ratio 
(SNR)  of  the  output  correlation  [42] .  Many  different 
expected  degradations  can  occur  between  the  input  and 
reference  image.  In  Fig.  10(c)  we  consider  rotational  misalign¬ 
ments  between  the  key  object  as  it  appears  in  the  input  and 
reference  scenes  and  the  effect  that  selection  of  the  proper 
/*  band  can  have  on  the  SNR  of  the  output  correlation.  As 
shown,  setting  K  =  1  at  a  large  /*  (band  H)  results  in  a  larger 
SNR  than  lower  f*  choices.  However,  as  the  angle  8  of  rota¬ 
tional  misalignment  between  the  input  and  reference  image 
increases,  the  SNR  of  the  correlation  with  the  MSF  formed 
with  K  =  2  in  band  H  rapidly  decreases.  Conversely,  a  lower 
f*  choice  (band  B)  results  in  a  lower  initial  SNR,  but  a  far 
lower  SNR  loss  with  8  variations. 

Thus  when  image  degradations  are  present,  lower  f*  settings 
are  preferable.  The  reason  is  simply  that  when  emphasizing 
high-input  spatial  frequencies  (by  high  /•  settings),  these  are 
the  first  components  lost  when  degradations  occur  between 
the  input  and  reference  imagery.  By  emphasizing  lower  input 
spatial  frequency  components  (by  lower  /*  settings)  and 
sacrificing  some  initial  SNR  and  Ip  (Fig.  10(c)),  we  find  the 
resultant  correlation  preferable  when  differences  or  degrada¬ 
tions  are  expected  between  the  input  and  reference  images. 
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Fig.  10.  Optical  pattern  recognition  by  weighted  MSF  control  |42], 
(a)  Input  scene,  (b)  l„  versus  /*  for  input  regions  A,  B,  C.  (c)  SNR 
versus  6  for  various  /*ri>ands. 

Many  types  of  image  differences  have  been  examined  [51] 
such  as  scenes  of  the  same  area  taken  years  apart,  in  different 
seasons,  from  different  aspect  angles,  and  radar  images  taken 
from  different  headings  or  from  different  sensors,  etc.  The 
magnitude  of  the  Ip  and  SNR  correlation  loss  for  each  of  these 
degradation  sources  were  found.  For  the  imagery  analyzed, 
textural-image  differences  (snow  cover  in  winter  scenes  and 
new  structure  in  imagery  taken  years  apart)  were  found  to 
have  less  of  a  degrading  effect  on  the  correlation  than  geo¬ 
metrical  image  distortions  (due  to  heading  or  aspect  differ¬ 
ences).  Weighted  MSF  synthesis  control  was  found  to  be 
helpful  in  reducing  the  correlation  loss  due  to  geometrical- 
image  differences  if  the  spectrum  of  the  imagery  was  broad 
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Fig.  1 1 .  Multisensor  imagery  exhibiting  intensity  disparity  [ 5 S | . 


and  if  the  image  degradation  could  be  classified  by  a  narrow¬ 
band  distortion  function.  Aspect  differences  satisfy  these 
criteria,  whereas  slant  range  error  and  radar  shadow  effects  in 
the  SLAR  imagery  examined  (opposite  headings  and  low 
depression  angles)  were  of  such  a  broad-band  nature  that 
extensive  compensation  for  correlation  loss  was  not  possible. 

Another  expected  source  of  image  differences  arises  when 
images  taken  from  different  sensors  are  correlated  [52] .  In  a 
missile  guidance  application,  the  stored  reference  image  is 
usually  assumed  to  be  a  high-quality  aerial  image  and  the  input 
image  a  real-time  radar  map.  In  such  cases,  intensity  differ¬ 
ences  occur  in  various  image  regions  between  the  two  scenes. 
The  intensity  of  a  given  object  depends  on  the  nature  of  the 
object,  the  illuminating  wavelength,  and  so  many  other 
factors  that  one  cannot  predict  in  advance  how  a  given  scene 
will  appear  in  a  radar  image  without  a  priori  knowledge  of 
the  content  of  the  various  regions  of  the  scene.  We  refer 
to  this  aspect  of  multisensor  image  processing  as  intensity 
disparity  image  pattern  recognition.  Similar  intensity  differ¬ 
ences  arise  between  imagery  taken  from  the  same  sensor  at 
different  times. 

One  of  the  classic  approaches  to  intensity  disparity  is  to 
differentiate  both  images  [53J .  The  premise  here  is  that  only 
the  edges  of  the  regions  in  such  scenes  are  the  features  that  are 
common  between  such  imagery.  However,  this  is  a  purely 
statistical  contribution  to  the  correlation.  This  contribution 
decreases  as  we  increase  the  number  N  of  uniquely  shaped 
regions  in  the  image  with  random  intensity  variations.  There 
are  also  various  types  of  image  differentiation  (54],  If  we 
consider  an  MSF  in  which  K  is  set  to  1  (optimum  modulation) 
at  a  high  spatial  frequency  / *  (thus  emphasizing  the  high 
spatial  frequency  portion  of  the  input  image),  we  are  effec¬ 
tively  performing  one  type  of  image  differentiation.  A  plot  of 
the  transmittance  of  such  an  MSF  shows  that  it  represents  a 
high-pass  filtered  transfer  function  with  a  6-dB  cutoff  at  the 
/*  spatial  frequency  at  which  K  was  set  to  unity  during  MSF 
synthesis. 

Thus,  in  performing  the  correlation  g  ®  h  using  an  MSF,  the 
reference  scene  h  can  be  modified  to  be  a  high-pass  filtered 
version  h'  by  the  action  of  the  MSF  (and  so  is  g).  The  corre¬ 
lation  g'  ®  h'  of  the  input  and  a  high-pass  filtered  reference 
function  is  thus  performed.  By  varying  /*,  we  can  control 
the  degree  of  image  differentiation  used  and  arrive  at  the 
optimum  combination  of  statistical  and  dete: ministic  (edge- 
enhanced)  pattern  recognition.  An  example  of  one  set  of 
multisensor  imagery  is  shown  in  Fig.  1 1 .  The  left  portion  of 
the  image  exhibits  several  fields  of  various  degrees  of  random 


intensity  variations  and  of  somewhat  unique  shapes.  This 
portion  of  the  image  can  be  viewed  as  a  set  of  N  key  objects 
(where  N  is  somewhat  small),  whereas  the  remaining  portion 
of  the  image  contains  little  information  content.  By  the 
proper  choice  of  the  spatial  frequency/*,  we  have  successfully 
correlated  all  four  images  in  this  and  similar  image  sets  using 
only  one  MSF  formed  from  one  of  the  images.  Similar  multi- 
spectral  correlations  on  various  other  image  sets  (including  a 
completely  contrast-reversed  image  pair)  have  been  achieved 
by  this  method  [55  J . 

VI.  Space -Variant  Pattern  Recognition 

As  noted  in  Sections  IV  and  V,  geometrical  differences  be¬ 
tween  the  input  and  reference  imagery  severely  degrade  correla¬ 
tions.  Scale  and  rotation  are  the  most  common  geometrical  dif¬ 
ferences.  These  can  be  accommodated  by  use  of  multiple  MSF’s 
at  Pi  of  Fig.  5 .  However,  this  approach  requires  that  multiple 
output  correlation  planes  be  scanned  and  always  results  in  a 
considerably  reduced  output  light  intensity.  In  another  ap¬ 
proach  to  producing  a  scale-invariant  optical  correlator,  the 
input  plane  P j  in  Fig.  5  is  placed  behind  Lx.  A  scaling  cor¬ 
relator  results  in  which  a  scale  search  of  the  input  function 
can  be  performed  by  varying  the  distance  from  Px  to  P-i . 
Rotational  differences  in  the  orientation  of  the  MSF  can  also 
be  accommodated  by  rotating  Px  or  Pi-  In  both  of  these 
latter  schemes,  the  time  required  to  manually  search  scale 
and  rotational  differences  is  not  compatible  with  the  real-time 
processing  advantages  of  an  optical  pattern-recognition  system. 
In  this  section,  we  consider  a  real-time  optical  processing  ap¬ 
proach  to  the  problem  of  geometrical  differences  in  pattern 
recognition.  The  resultant  optical  processor  will  be  a  space 
variant  rather  than  a  space-invariant  system. 

One  of  the  major  limitations  in  the  realization  of  a  practical 
recognition  system  has  been  the  space  invariance  of  an  optical 
processor.  This  feature  allows  these  systems  to  be  described 
by  the  convolution  integral  in  (10).  With  reference  to  Fig.  5, 
the  input  function  is  g,  the  system’s  impulse  response  is  h 
(the  transform  of  the  system’s  transfer  function  H  recorded  at 
Pi ),  and  the  content  of  the  output  plane  P3  pattern  is  the  con¬ 
volution  of  g  and  h.  Considerable  research,  presently  in  prog¬ 
ress,  is  concerned  with  producing  various  types  of  space-variant 
optical  systems  (whose  impulse  response  varies  spatially  with 
input  position).  One  program  in  this  area  directly  related  to 
pattern  recognition  involves  the  use  of  coordinate  transforma¬ 
tion  preprocessing  [56]  as  initially  described  by  Huang  [57] 
and  Sa  wchuck  [  58] .  We  describe  two  examples  of  such  optical 
systems  in  detail  below. 

We  consider  first  a  space-vanant  optical  pattern-recognition 
system  whose  output  correlation  is  invariant  to  scale  differ¬ 
ences  between  the  input  and  reference  functions.  By  applying 
a  logarithmic  transformation  to  the  input  and  reference  func¬ 
tions,  we  obtain  (in  1  -D  for  simplicity)  g'  =  g(exp  x)  and  h'  - 
A(exp  x).  The  FT  of  these  functions  yields  G'  and  H'  (or  H'*). 
It  has  been  shown  [56]  that  the  FT  ofg’  is  the  Mellin  trans¬ 
form  1 59]  of  g.  Of  particular  interest  is  the  case  when  h(x)  = 
g{ax)  (i.e.,  g  and  h  differ  by  a  scale  factor  a).  In  this  case  the 
Mellin  transforms  Mg  and  Mh  of  g  and  h  are  related  by 

Mg(u)  -  Mh(u)  exp  (-/2tru  In  a)  exp  (-/2naxj).  (21 ) 

This  follows  from  the  definitions  of  Mg  and  as  the  FT’s  of 
g  and  h’.  The  final  exponential  factor  in  (21)  is  due  to  the 
spatial  carrier  a  on  which  is  recorded  (as  in  the  MSF  syn¬ 
thesis  and  correlation  operation  described  in  Section  IV). 
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When  realized  using  the  FPC  system  of  Fig.  5 ,g  is  placed  at 
Py  and  AfjJ  is  recorded  at  Pi  of  Fig.  S.  The  light  incident  on 
Pi  is  Mg  (the  FT  of  g')  and  the  light  distribution  leaving  Pi  is 
MgMj,.  Lens  L2  in  Fig.  5  forms  the  FT  of  MgMg  at  P3  where 
we  find 

u3  =  SlMgMf,)  =  5[MhMh  exp[-/2ff(u  In  a  +  ax2)l 

=  h  ®  h  *5(xj  -  lnaa\/L).  (22) 

From  (22),  we  see  that  the  resultant  correlation  is  displaced 
from  its  normal  location  x3  =  a\fL  by  an  amount  proportional 
to  the  natural  logarithm  of  the  unknown  scale  factor  “a”  be¬ 
tween  the  two  functions.  Thus  from  the  location  of  the  cor¬ 
relation  peak  we  can  determine  “a.”  Of  more  importance  is 
the  fact  that  the  output  correlation  is  identical  to  the  auto¬ 
correlation  h  ®  h,  just  as  if  g  =  h  with  no  scale  difference 
present. 

We  now  consider  a  space-variant  optical  pattern- recognition 
system  that  is  invariant  to  rotational  differences  between  the 
input  and  reference  images.  The  same  coordinate  preprocess¬ 
ing  approach  is  again  used.  The  two  functions  are  h  and  g  (a 
rotated  version  of  h  with  rotational  angle  0O).  We  first  convert 
both  h( x,  y)  and  g(x,  y)  to  new  functions  h'(i-,  ij)  and  g'(J,  r? ) 
by  the  coordinate  transformations  £  =  tan (y/cc)  and  13  = 
\Jx2  +y 2  (i.e.,  a  polar  transform).  This  transformation  now 
converts  a  rotation  by  0O  into  a  shift.  The  1  -D  conjugate  trans¬ 
form  H’*  of  the  coordinate  transformed  undistorted  function 
h'  is  formed  and  stored  at  P2  of  Fig.  5. 

With  g'  recorded  at  Pt  of  Fig.  5,  the  light  distribution  leaving 
P2  is  G’H'*.  To  best  understand  how  the  rotational  invariance 
is  achieved,  we  separate  the  original  function  g  into  two  parts 
gi  and  g2  (where  in  polar-coordinates,  g  occupies  the  0  to 
-0O  portion  of  0  space  and  g2  occupies  the  remaining  0  to 
2 it  -  0O  region).  The  portions  G  [  and  G2  of  the  FT’s  of  the 
coordinate-transformed  polar  functions  g'(ij,  rj )=g'(0,  r)  are 
related  to  the  FT’s  G 1  and  G2  of  the  corresponding  portions 
gy  and  g2  of  the  original  g(x, y)  image  by 

G'  =  G[  +  G2  =  Gy  exp(-/coe0o) +G2  exp  [-/w8(27T  -  0o)J 

(23) 

where  log  denotes  the  Fourier  plane  spatial  frequency  variable 
associated  with  the  polar-angle  variable  0  =  ?. 

The  light  leaving  P2  is  now  G'H'*  and  its  FT  formed  at  P3  of 
Fig.  5  by  l2  is 

u3  =  5[G'H'*  1  =  g[  ®  h  +  g2  ®  h 

=  h,  ®  h  *  b(8  -  d0)  +  h2  ®  h 

*5(0 +2tr-0o).  (24) 

From  (24),  we  see  that  the  output  correlation  is  now  divided 
into  two  correlation  peaks  separated  in  distance  by  2tr  in  P3 
space.  The  sum  of  these  two  correlation  peaks  is 

u3  =  hy  ®  h  +h2  ®  h  =  h  ®  h  (25) 

or  the  output  correlation  of  two  rotated  functions  equals  the 
autocorrelation  of  the  original  unrotated  functions.  There  is 
thus  no  correlation  loss  even  if  the  input  and  reference  func¬ 
tions  are  rotated  versions  of  one  another  with  an  unknown 
rotational  angle  0O.  From  (24)  we  see  that  the  location  of 
the  output  correlation  peak  is  proportional  to  the  unknown 
rotational  angle  0O  and  thus  0O  can  also  be  found  from  the 
output  correlation . 
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Fig.  12.  Rotation-invariant  optical  pattern  recognition.  (Top)  input. 
(Center)  output  correlation  plane.  (Bottom)  cross-sectional  scan  of 
the  correlation  peak. 

An  example  of  this  rotation  invariance  is  shown  in  Fig.  1  2. 
The  reference  image  chosen  was  an  airplane,  an  MSF  of  the 
polar  transformed  version  of  the  airplane  was  recorded  at  P2 
of  Fig.  5.  As  the  input  was  rotated  (Fig.  12,  top),  its  polar 
transform  was  formed  and  recorded  at  /*, .  The  resultant  out¬ 
put  correlation  plane  pattern  (Fig.  12,  center)  and  its  cross 
sectional  scan  (Fig.  12,  bottom)  show  the  predicted  results. 
The  location  of  the  output  correlation  peak  changes  propor¬ 
tional  to  the  rotational  misorientation  between  the  input  and 
reference  and  the  output  correlation  peak  remains  essentially 
invariant  to  the  input  rotational  differences. 

The  log  coordinate  transformation  required  in  the  Mellin 
transform  correlator  can  easily  be  realized  by  use  of  log  mod¬ 
ules  in  the  deflection  system  of  the  input  plane  SLM.  The 
polar  transforms  in  Fig.  1 2  were  realized  in  real  time  using  a 
specially  modified  camera.  Extensions  of  these  optical  space- 
variant  pattern-recognition  methods  and  the  formulation  of  a 
system  invariant  to  multiple  distortions  are  described  in  detail 
elsewhere  [60] . 

VII.  Hybrid  Optical/Digital  Correlators 
The  final  embodiment  of  any  optical  processor  will  be  a 
hybrid  system  [36]  in  which  the  best  features  of  optical  and 
digital  processing  are  properly  married.  The  architecture  of 
one  such  hybrid  system  that  has  been  assembled  [61 1  is  shown 
in  Fig.  13.  The  upper  portion  of  this  schematic  shows  the 
FPC  system  of  Fig.  5  with  real-time  SLM’s  at  the  input  and 
filter  planes.  Digital  control  of  the  content  of  these  SLM’s 
and  control  of  the  system’s  write,  read,  and  erase  cycle 
are  implied  in  the  flowlines  shown.  The  digital  preprocessor 
in  Fig.  1 3  is  used  for  format  control  and  as  a  scan  converter. 
As  shown,  the  outputs  at  the  FT  and  correlation  planes  are 
detected  by  a  2-D  array  and  analyzed  by  a  microprocessor 
array  (250-ns  cycle  time,  Intel  3000  series)  to  extract  the 
desired  information  (usually  the  location  of  the  correlation 
peak).  A  buffer  memory  within  the  interface  allows  interframe 
operations  to  be  performed  such  as  change  detection  and  out¬ 
put  time  history  of  correlation  peaks  to  be  found. 

The  microprocessor-interface  system  allows  us  to  digitize  the 
output  correlation  plane  to  a  variable  resolution.  This  is  an 
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Fig.  13.  Schematic  diagram  of  a  microprocessor-based  hybrid  optical/ 
digital  pattern-recognition  system  (61). 


important  feature  as  it  allows  us  to  reduce  the  quantity  of  out¬ 
put  data  to  be  analyzed,  as  the  application  warrants.  Simple 
thresholding  of  the  output  plane  to  any  of  256  gray  levels  is 
possible.  However,  such  a  simple  detection  scheme  is  rarely 
adequate.  Thus  an  integrator,  peak  detector,  and  counter  were 
added  to  the  output- detection  system  to  allow  use  of  alternate- 
detection  criteria  such  as  the  area  under  the  correlation  peak 
or  whether  the  output  signal  exceeds  a  given  threshold  level 
for  a  given  amount  of  time.  We  have  shown  earlier  [42]  that 
the  area  of  the  correlation  peak  decreases  less  rapidly  than  Ip 
with  rotation,  scale,  or  other  distortion  parameters.  Thus  use 
of  such  alternate  detection  methods  is  another  way  by  which 
the  pattern-recognition  system’s  sensitivity  to  differences  be¬ 
tween  the  input  and  reference  imagery  can  be  reduced. 

An  alternate  version  of  a  hybrid-correlator  can  be  realized 
by  placing  a  TV  or  linear  CCD/photodiode  detector  in  the  fre¬ 
quency  plane  P2  of  the  JTC  system  of  Fig.  7.  Lens  Li  and 
plane  P3  in  Fig.  7  are  not  used  now,  rather  the  transform  of 
the  data  at  P7  previously  performed  by  L2  is  now  realized 
by  a  digital  fast  Fourier  transform  (FFT)  or  spectrum-analyzer 
system  [62]-[64] . 

To  accomplish  the  processing  previously  performed  by  L2  in 
Fig.  7  on  the  joint- transform  pattern  of  (17),  the  pattern  in 
(18)  is  detected  and  scanned  as  a  time  or  video  signal  v(t).  An 
FT  operation  on  tt(f)  is  realized  by  multiplying  v(t)  by  quadra¬ 
ture  sinusoids  (sin  oj0t  and  cos  oJoOi  the  two  separate  products 
are  then  integrated  over  one  line  scan  time  of  the  output  de¬ 
tector,  squared,  and  summed.  From  (18),  we  see  that  the 
|G|2  +  \H\2  envelope  is  the  slowly  varying  part  of  the  output 
video  signal,  whereas  the  important  part  of  u(t)  is  the  cosine 
function. 

If  the  input  and  reference  patterns  are  equal,  g  =  h  and  ( 1 8) 
becomes 

«>(/)  =  2|//|J(1  +  cos4»r6u).  (26) 

For  this  case  the  frequency  of  the  cosine  depends  upon  the 
separation  between  the  input  and  reference  in  P j  or,  therefore, 
the  location  of  h  in  g.  If  g  is  displaced  from  h  by  2.5 6  rather 
than  26,  then  (26)  becomes 

n(f)  =  2 1 // 1 2 1 1  +  cos  (4»r6u  +tr6u)]  (27) 

from  which  we  see  that  the  location  of  h  in  g  can  be  found 
from  the  frequency  of  the  cosine  pattern. 

To  demonstrate  this  principle,  a  landing  field  was  chosen  as 
the  key  reference  object  h  to  be  located  (Fig.  14(a),  left)  in  a 
larger  input  scene  (Fig.  14(a),  right).  Several  of  the  input 
image  pairs  at  Pi  of  the  ITC  system  are  shown  in  Fig.  14(a), 
with  the  separation  between  h  and  g  in  the  two  images  increas¬ 
ing.  The  joint  transform  of  each  input  image  pair  was  de- 


(b) 

Fig.  14.  Experimental  demonstration  of  a  hybrid  optical  coherence 
measure  pattern-recognition  system  |64j.  (a)  Inputs,  (b)  Cross- 

sectional  scan  of  the  output  correlation. 

tected,  scanned,  and  quadrature  modulated  as  noted  earlier. 
As  the  frequency  of  the  local  oscillator  was  swept  with  time, 
the  output  was  displayed  (Fig.  14(b)).  The  horizontal  axis 
in  Fig.  14(b)  thus  corresponds  to  different  mixing  oscillator 
frequencies  or  equivalently  to  testing  different  displacements 
or  shifts  of  the  input  function.  As  shown,  the  location  of 
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the  peak  output  from  the  electronically  processed  joint- 
transform  pattern  shifts  right  as  the  location  of  h  in  g 
shifts  right.  The  presence  of  a  peak  indicates  the  presence  of 
h  in  g  and  the  location  of  the  peak  indicates  the  location  of  h 
in  g. 

Returning  to  (26)  and  (27),  we  see  that  the  amplitude  of  the 
recorded  cosine  pattern  depends  on  the  product  of  the  magni¬ 
tudes  of  the  FT’s  of  the  two  input  functions.  If  g  and  h  are 
identical  except  for  their  strength,  i ,e.,g(x)  =  a/i(x)  in  1  -D,  the 
detected  pattern  is 

ti  =  |f/|J(l  +a  +  2a  cos4frt>u).  (28) 

From  (28),  we  find  that  the  modulation  is  reduced  propor¬ 
tional  to  the  reduced  signal  strength  of  g.  Since  the  amplitude 
2a  of  the  cosine  term  can  easily  be  found  from  the  electroni¬ 
cally  processed  output,  the  vertical  scale  in  the  output  pattern 
(Fig.  14(b))  can  appropriately  be  adjusted  and  thus  a  normal¬ 
ized  correlation  produced. 

Another  feature  of  electronic  postprocessing  and  further  in¬ 
sight  into  the  operation  of  the  original  JTC-system  can  be  seen 
by  considering  the  case  when  h  g.  The  fringe  pattern  at  P- > 
is  now  no  longer  a  pure  cosine  but  is  phase  modulated  (spatially) 
by  the  difference  between  the  phases  of  the  transforms  G  and 
H.  The  power  spectral  density  at  the  carrier  frequency  2b  is 
now  decreased  proportional  to  the  degree  of  mismatch  of  g 
and  h.  Thus  from  the  magnitude  of  the  output-spectral  density, 
we  have  a  measure  of  the  spectral  purity  of  the  video  signal 
and  equivalently  the  coherence  or  similarity  of  the  two  inputs. 
The  carrier  period  variance  and  carrier  phase  variance  can 
easily  be  found  (e.g.,  from  a  phase-locked  loop  on  the  output) 
and  hence  a  “coherence  measure  correlator”  results. 

However,  electronic  postprocessing  also  allows  increased 
flexibility  since  the  signals  can  be  modified  prior  to  quadrature 
modulating  and  processing  them.  This  luxury  and  flexibility  is 
not  present  in  the  full  optical  system  of  Fig.  7,  since  Lt  must 
take  the  FT  of  the  P 2  pattern  (the  joint  transform  of  G  and 
H )  and  once  detected  Pi  cannot  easily  be  modified.  In  prac¬ 
tice  g  and  h  will  differ  as  noted  in  Sections  IV  and  V.  For  the 
case  when  the  distortion  present  in  g  can  be  described  by  the 
action  of  a  linear-space  invariant  system  with  impulse  response 
/,  we  can  describe  g  by  g  =  h  */.  In  many  instances  the  dis¬ 
tortion  /  is  known  or  reasonable  distortion  functions  can  be 
assumed  a  priori.  In  such  cases,  the  electronic  postprocessor 
of  the  Pi  data  can  be  modified  (for  example,  by  phase- 
modulating  the  local  oscillator  in  the  detection  system  with 
the  transfer  function  F  of  the  assumed  distortion  function  /). 
An  “equalizing  correlator”  (a  term  borrowed  from  communi¬ 
cations)  then  results  in  which  various  distortions  present  in  g 
that  are  not  in  h  can  be  removed  (or  their  effects  equalized). 
Experimental  confirmation  of  the  “equalizing  optical  correla¬ 
tor”  has  also  been  reported  [64] . 

An  alternate  technique  by  which  improved  discrimination 
can  be  achieved  in  optical  pattern  recognition  was  suggested 
by  Caulfield  and  Maloney  (65).  In  this  approach,  linear  com¬ 
binations  of  the  system’s  responses  to  various  input  characters 
are  formed  in  a  second  processing  stage. 

Electronic  preprocessing  and  postprocessing  techniques  have 
recently  been  used  to  enable  bipolar  correlations  to  be  per¬ 
formed  on  incoherent  processors  (661 -(69).  The  use  of 
computer-generated  holograms  to  synthesize  spatial  filters 
(70),  (71]  of  functions  not  easily  physically  realizable  repre¬ 
sents  yet  another  use  of  digital  techniques  in  optical  pattern 
recognition. 


VII.  Summary 

Although  many  survey  articles,  special  issues,  and  books 
exist  on  optical  computing,  this  article  is  the  first  to  address 
only  the  issue  of  optical  pattern  recognition -thus  allowing 
a  unified  treatment  in  some  depth.  When  we  think  of  optical 
pattern  recognition,  we  usually  consider  only  the  optically 
matched  spatial-filtering  system  of  1964.  However  as  shown, 
many  advances  have  occurred  since  1 964.  A  wealth  of  new 
architectures  for  optical  pattern-recognition  systems  exist  and 
have  been  demonstrated  and  many  refinements  have  occurred 
in  the  original  Vander  Lugt  correlator. 

Research  in  optical  pattern  recognition  is  now  addressing 
more  practical  and  real  pattern-recognition  problems.  These 
directed  research  efforts  and  major  advances  in  optoelectronic 
and  solid  state  components,  plus  advances  in  digital  electronics 
and  digital  pattern- recognition  algorithms  have  affected  optical 
pattern-recognition  systems.  A  rapidly  increasing  repertoire 
of  new  operations  (such  as  space-variant  ones)  are  now  possible 
in  optical  processors.  Many  of  these  have  already  been  applied 
to  pattern-recognition  applications. 

Many  other  optical  processing  operations  not  discussed  in 
this  survey  exist  that  do  not  use  the  classic  FT  property  of  a 
lens.  These  include  an  optical  system  to  realize  the  discrete 
FFT  (72],  optical  systems  operating  in  residue  arithmetic 
(73] -175]  and  many  others  (11],  In  time,  these  and  other 
optical  processing  methods  will  be  applied  to  pattern- 
recognition  applications.  This  will  further  increase  the  capa¬ 
bility  of  optical  pattern-recognition  systems,  while  retaining 
the  high-speed  parallel  processing,  and  large-space-bandwidth 
product  features  of  optical  processors  that  have  made  them 
attractive  for  the  past  twenty  years. 
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